Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

Backing Up And Restoring Linux With SystemImager
(Open source abandonware)

News

Simple Unix Backup Tools

Recommended Books Recommended Links Bright Cluster Manager rsync RPM Packages

GNU ddrescue

Yast

 

Swap partition

Siga

Installing Suse from ISO image

Humor Etc

NOTE: This is an interesting project which now is abandonware (last changes are from 2015). Like many such projects it died of overcomplexity.  It can be simplified and revived without bell and wiles as key ideas are sound and there are commercial systems like Bright cluster manager which use the same approach (rsync based restoration os nodes)

SystemImager was a popular tool created by Brian Elliott Finley,  while he was working in Argonne National Laboratory. The name "System Imager"  is trademarked.  Systemimager is licensed under GPL v 2.  Now this is an open source abandonware.

Source code is now available from GitHub - finley-SystemImager.  Key utilities are

As it is based on rsync SystemImager is distribution-agnostic and that means that it can support heterogeneous set of nodes (aka grid). In other words it allows the deployment of any Linux distribution (standard or even customized) to any kind of target machine as long as you can providea bootable image for this machine. Typical environments include computer labs and render farms, but it has proven particularly popular in clustered computing environments, such as grid and high performance computing.

The main goal of the project was to make deployment of large numbers of computers easy, but trying to achive that they went of the couse and make things way too complex. 

Rsync works with files based (rather than disk blocks) so SystemImager images are stored as a directory hierarchy of files representing a comprehensive snapshot of a machine. This snapshot is containing all the files and directories from the root of the server file system.

Images can be acquired in multiple ways. The standard method of image creation involves cloning of a pre-installed machine, so called the golden-client. In this way, the user can customize and tweak the golden-client's configuration according to his needs, verify it's proper operation, and be assured that the image, once deployed, will behave in the same way as the golden-client. Incremental updates are possible by syncing an updated golden-client to the image, then syncing that image to deployed machines using the si_updateclient command.

Images are hosted in a central repository on a server, called the image-server, and they can be distributed among the clients using several different transport mechanism: rsync
(the default), multicast (via Flamethrower1), SSL encrypted rsync (using a SSH tunnel), and (for a very large set of nodes) via BitTorrent.

Commercial implementation of the same ideas as SystemImager can be found in Bright Cluster Manager.  Both rely on PXE boot, which is not available in all environments.

To implement PXE boot you need to provide two critical peaces of infrastructure (see  Chapter 30. Setting Up an Installation Server )

  1. TFTP server
  2. DHPC server

If you do not control DHCP server you only option is to use private network, which is a common solution for HPC clusters, but has little value outside this niche.

Some ideas about how to resurrect SystemImager and make it simpler

This is too valuable abandonware to make it die.

A simpler implementation is to create initial install using the kickstart, then duplicate it into RAM drive, chroot to the duplicate and overwrites  the installed root filesystem with the image. that exclude the necessity of DHCP server, PXE, etc. Or more correctly to cut the complexity it make sense to make them independent.

In other words systemimager can be re-implemented as the kickstart post install script.  That limits the number of distribution it supports.

Idea of overrides

System imager supports 3 levels of hierarchy for the overrides over the image:

Implementation

SystemImager is mostly written in Perl. the problem is that in the current form it makes use of 18 packages which makes implementation rather complex. The current implementaion of SI uses the folowing packages:  

  1. busybox

  2. bc

  3. devfsd

  4. ISC dhcp

  5. discover

  6. dosfstools

  7. e2fsprogs

  8. jfsutils

  9. xfsprogs

  10. Linux kernel

  11. parted

  12. pxelinux

  13. rsync

  14. syslinux

  15. raidtools

  16. reiserfsprogs

  17. systemconfigurator

  18. uClibc

Implementation via kickstart will remove almost all of them.

Later it became a part of System Installation Suite (SIS), which includes SystemInstaller, SystemImager, and System Configurator.  Of them only System Configurator is still maintained.

SystemInstaller is a tool that allows you to install images directly to a SystemImager image server. System Configurator, which is also used by the standard SystemImager release, performs configuration of target machine uniquenesses such as IP addresses, network cards, and initial RAM disks needed to boot clients after installation.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

Package Repository finley-SystemImager Wiki GitHub

Brian Elliott Finley edited this page on Oct 2 2016 4 revisions Simple style -- just copy and paste this one-liner to install the repo config file:
curl -s http://systemimager.org/install-repo.sh | sudo sh

Alternate Method

Choose the appropriate chunk below, then scrape and paste the entire chunk at once. No need to do one line at a time.

Ubuntu and Debian

# choose either the stable or testing repo
REPO=stable
REPOFILE=systemimager.org-$REPO.list
URL=http://download.systemimager.org/sources.list.d/$REPOFILE
sudo wget $URL -O /etc/apt/sources.list.d/$REPOFILE
wget http://systemimager.org/pub/brian@thefinleys.com.gpg.key -O - | sudo apt-key add -

Red Hat, CentOS, and friends

# choose either the stable or testing repo
REPO=stable
REPOFILE=systemimager.org-${REPO}.repo
RELEASE=$(uname -r | sed -r -e 's/.*((el|fc)[0-9]+).*/\1/')
URL=http://download.systemimager.org/pub/yum.repos.d/$REPOFILE.$RELEASE
cd /etc/yum.repos.d/
curl --url $URL --output $REPOFILE
rpm --import http://systemimager.org/pub/brian@thefinleys.com.gpg.key 

Browse the Repo

http://systemimager.org/pub/

Recommended Links

Softpanorama hot topic of the month

Softpanorama Recommended

Index of -pub-systemimager

[   ] systemimager-datasheet-1.5.0.pdf 07-Aug-2015 17:45 113K  
[   ] systemimager-manual-v4.1.6.pdf 07-Aug-2015 17:10 472K  

 

Package Repository finley-SystemImager Wiki GitHub

SystemImager - Wikipedia

SystemImager CF TWiki

SystemImager and BitTorrent: a peer-to-peer approach for Large-Scale OS Deployment