Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-) Softpanorama Search

 Backup and Restore

The traditional UNIX® backup programs are dump and restore. They operate on the drive as a collection of disk blocks, below the abstractions of files, links and directories that are created by the file systems. dump backs up an entire file system on a device. It is unable to backup only part of a file system or a directory tree that spans more than one file system. dump does not write files and directories to tape, but rather writes the raw data blocks that comprise files and directories.

• YaST contains a backup applet that uses tar for creating backups. That means that it is not system backup as tar does not correctly backs up links and devices.  It will not back up your entire drive, but it can get all the critical files that can help to restore you system after you reinstalled the OS. You can use this tool to create several backup profiles, depending on your strategy. The System Backup and Restore System tools are both located in the YaST System page.  The first time you start System Backup, you will get a blank page. You must first create a backup profile.
   
• dd The device-to-device (dd) utility is used to copy a file from one device to another. It goes beyond that in functionality, however, for it can convert a file during the copy process from one format to another. It can convert from EBCDIC to ASCII (and reverse), change uppercase to lowercase (and reverse as well), and work with bytes, blocks, or keywords.

  The most common use for dd is copying files to and from removable media, and you must use arguments that can include

  • bs— block file size

  • if— input file

  • of— output file
     

• tar This program, which is a standard part of all major Linux distributions, is a simple but popular backup tool. It's described in more detail in the next section. This program performs backups and restores on a file-by-file basis, placing all files in a carrier file. It's also frequently used to create tarballs, which are disk-based archives of files that can be moved across a network, placed on removable media, and so on. Tarballs are commonly used to distribute program source and executable files.
   
• cpio The cpio program is conceptually similar to tar, in that it's a file-by-file backup tool that creates an archive file. This file can be compressed or copied to a backup medium.

• dump The dump program is another file-by-file copying program; however, dump is tied to a specific filesystem, such as ext2fs or XFS. It reads filesystem data structures at a lower level than tar or cpio, and can therefore back up files in a slightly less intrusive way. Unfortunately, versions of dump are not available for all filesystems; in 2004, only ext2fs/ext3fs and XFS have dump programs, of common Linux filesystems. Worse, with 2.4.x and later kernels, dump may not work reliably, so it shouldn't be used. (See http://lwn.net/2001/0503/a/lt-dump.php3 for a mailing list message from Linus Torvalds on this subject.) To restore data backed up using dump, you must use a separate restore program.
   

• Partition Image This program works at a still lower level than dump; instead of backing up individual files, it backs up disk sectors that are marked as being used. This method of operation means that Partition Image is tied to the filesystem you use. As of Version 0.6.4, stable filesystems are ext2fs/ext3fs, ReiserFS, JFS, XFS, FAT, and HPFS. UFS and HFS are considered beta, while NTFS support is marked as experimental. This package can only back up and restore an entire partition, which makes it most useful for creating images of just-installed desktop systems and the like, rather than backups from which individual files might need to be retrieved in the future. You can learn more at http://www.partimage.org.

In the days when the programs dump and restore were the preferred backup solution, many system administrators would take the filesystems offline (that is, unmount them) before backing up. There was a risk that dump would fail if a live filesystem were updated during the dump. This technique was possible because dump opened the raw device (such as /dev/hda1), and decoded the filesystem structure directly, rather than going through the usual system-call interface to access the directories and files. This technique won't work for tools like tar, which (like any other normal application) require a filesystem to be mounted before its contents can be accessed. There is a small risk that a file will end up corrupt on a tar archive if it happens to get updated while tar is in the middle of reading it. If this risk is not acceptable, one option is to bring the system down to single-user mode, so that the filesystems are quiescent.

If your filesystem is on a logical volume, another option is to take a snapshot of the volume, then archive the snapshot.You can create a new logical volume that contains a snapshot of this filesystem with the commands:

# modprobe dm-snapshot
# lvcreate 
 -L 512M -s -n ftpsnap /dev/system/ftp

Here, the modprobe command was necessary to load the device mapper snapshot module into the kernel (one of the few times I've found it necessary to load a module explicitly). The -s flag to lvcreate says to create a snapshot. The -n flag gives the snapshot logical volume a name (it will be created within the system volume group) and the final argument names the existing logical volume you want to take the snapshot of. After this, you have two logical volumes, like this:

# lvscan
  ACTIVE   Original '/dev/system/ftp' [512.00 MB] inherit
  ACTIVE   Snapshot '/dev/system/ftpsnap' [512.00 MB] inherit

Now you can mount the snapshot:

# mount /dev/system/ftpsnapshot /mnt

and use tar, for example, to safely archive the files from it.

Creating a snapshot logical volume does not just blindly copy the whole of the original volume into the snapshot—it's more intelligent than that. A snapshot contains only those blocks that differ from the original. As the original or the snapshot is written to, obviously the number of different blocks will grow.

Old News ;-)

[Nov 21, 2008] rsnapshot - Local-Remote Filesystem backups utility in openSUSE SUSE & openSUSE

Probably does not make much sense but Perl codebase might be reused.

Posted by admin on October 4th, 2008

rsnapshot is a filesystem backup utility based on rsync. Using rsnapshot, it is possible to take snapshots of your filesystems at different points in time. Using hard links, rsnapshot creates the illusion of multiple full backups, while only taking up the space of one full backup plus differences. When coupled with ssh, it is possible to take snapshots of remote filesystems as well.

rsnapshot is written in Perl, and depends on rsync. OpenSSH, GNU cp, GNU du, and the BSD logger program are also recommended, but not required. rsnapshot is written with the lowest common denominator in mind. It only requires at minimum Perl 5.004 and rsync. As a result of this, it works on pretty much any UNIX-like system you care to throw at it.

rsnapshot can run almost out of the box with very little configuration changes although advanced configurations can be done with little more effort.

SystemRescueCD backup partitions and drives

HowTo: Install SystemRescueCD on a Dedicated Hard Disk Partition.
OR
HowTo: No-effort Backup Solution for Partitions and Hard Drives

Introduction: Downloading and burning SystemRescueCD provides a bootable Gentoo-based distro on a CD. The installed applications focus on restoring disabled Linux/windows distros on the hard drives, or retrieving data if things go terribly wrong. You can operate direct from the booted CD. It's great. But SystemRescueCD also contains PartImage, the powerful free Linux alternative to Norton Ghost. So it's a too-easy tool for backing up single or multiple partitions, or whole drives.

Here I recount HowTo install SystemRescueCD onto a dedicated partition. I include a script for creating backup images of your hard drive partitions. Once you go through this tutorial as a practical exercise, you'll have the knowledge and confidence to customise all manner of backup solutions so very easily.

This tutorial is for the middle ground reader, too hard for new Linux users and too simple for Gurus. It's drawn from material in the On Line Manual at the System Rescue CD Site.  

Summary of the steps for installing SystemRescueCD on a dedicated hard disk partition:

1. Prepare a separate SystemRescue partition
2. Download the SystemRescueCD ISO file
3. Extract bootable image files from the ISO to the boot partitiion
4. Edit Suse's GRUB configuration to facilitate booting the SystemRescue partition
5. Prepare and place your scripts, if any
6. Boot with Suse's loader --> select item SystemRescueCd

Step 1: Prepare a separate SystemRescue partition: I leave it to you to make the partition. You need about 160Mb plus any extra storage you might need. I use 400Mb. Note that this partition becomes the root of the cdrom after SysRescueCD boots from it, so its filesystem becomes read-only. This means you will select writeable workspaces on other partitions.

Suppose for illustration that you have prepared partition hda13 for the installation. Now make a directory to mount hda13 into openSUSE, e.g. /SysRescCD. You can mount hda13 with this command:

mount /dev/hda13 /SysRescCD

BUT I use the more convenient permanent mount created by placing this line into /etc/fstab:

/dev/hda13    /SysRescCD    ext3    defaults    1 2

You can do that with Yast --> System --> Partitioner OR more simply by issuing this command in a console: kdesu kwrite /etc/fstab and then typing the line in.

Step 2: Download the SystemRescueCD ISO file: You can download the CD ISO for the SystemRescueCD by following this project download link. The ISO filename looks like this: systemrescuecd-x86-x.y.z.iso. Place it anywhere on your hard drives, at e.g. /path_to/systemrescuecd-x86-x.y.z.iso

Step 3: Extract bootable image files from the ISO and place them in boot partition: You can mount the ISO file for viewing the files on the CD. First create a folder to mount the ISO in, e.g. /iso. Then mount the ISO with this command in a root terminal:

mount -o loop -t iso9660  /path_to/systemrescuecd-x86-0.3.6.iso    /iso

You'll find these three files of special interest on these paths inside the mount folder:

 

/iso/sysrcd.dat

 

/iso/isolinux/rescuecd

 

/iso/isolinux/rescuecd.igz

Create the folder sysrcd in the root of hda13, using the mount point /SysRescCD to place it at /SysRescCD/sysrcd. Then copy the three files into folder sysrcd. The name sysrcd is immutable.

The partition hda13 is now configured with the bootable Gentoo distro and all that remains is to point Suse's bootloader at it.

Step 4: Edit Suse's GRUB configuration to facilitate booting the SystemRescue partition: You can open the Grub configuration file in a text editor with commands like this one for Kwrite:

kdesu kwrite /boot/grub/menu.lst

Edit/add these lines at the bottom of the file, one blank line below the last entry:

title    SystemRescueCd
root    (hd0,12)
kernel    /sysrcd/rescuecd root=/dev/ram0 init=/linuxrc looptype=squashfs loop=/sysrcd/sysrcd.dat splash=silent nosound subdir=sysrcd cdroot=/dev/hda13 setkmap=us vga=0x31a
initrd    /sysrcd/rescuecd.igz
boot

Remember to adapt my (hd0,12) which is for my hda13, across to your situation. Also, note that the sequence beginning "kernel" and ending "0x31a" is all the same/one line. I've included three parameters at the end: cdroot=/dev/hda13 setkmap=us vga=0x31a. These set the distro upto have hda13 at the root of the cd (on /mnt/cdrom), to have the US keyboard and for a vga screen that suits me. If you wanted to boot into the Window Manager Desktop Environment to access GUI tools, you would use this line instead:

kernel    /sysrcd/rescuecd root=/dev/ram0 init=/linuxrc looptype=squashfs loop=/sysrcd/sysrcd.dat splash=silent nosound subdir=sysrcd cdroot=/dev/hda13 setkmap=us vga=0 dostartx

A list of boot options can be seen on this link at the SystemRescueCD site.

Step 5: Prepare and place your scripts, if any: Pre defined sites [like the floppy disk, the root of the CDROM, the root of the installation partition] are searched straight after booting for scripts which if found are executed. You can have one or many scripts. See the SystemRescueCD site for full details. I'll deal with only one location here: the root of the installation partition. It's really simple. Rust create a script called autorun and lodge it in the root of the installation partition, hda13. It will run just after the system boots to a console.

Step 6: Boot with Suse's loader: Reboot and select item SystemRescueCd. The root of partition hda13 automounts at location /mnt/cdrom in the booted-up virtual filesystem. All files and scripts placed on the partition are thus available at /mnt/cdrom.

Backup Script: I constantly change the filesystems on my primary hard drive and it's hard to prevent damage to them. So I back them up regularly. This takes a long time. I use a script called autorun in the root partition of hda13 and simply boot to SystemRescueCD on hda13 and walk away to let the job proceed. Here's my scenario and script. You could easily modify the script for your scenario.

Scenario: I have a Suse root partition at hda5 and a /home partition at hda6. These have to be backed up when they're not being used, i.e. from within another operating system. The Gentoo installation on the SystemRescueCD partition contains a script, "autorun", which employs "partimage", the Linux free version of "Ghost". It is perfect for the task. I have prepared a folder called "partimage" on partition hdb2 on IDE2 drive. The script mounts hdb2 into Gentoo/SystemRescueCD's filesystem, generates a date-coded folder on hdb2 and copies image files across from the Suse root and home partitions.

Script

#!/bin/sh
# mount the target directory
mkdir /mnt/hdb2
mount /dev/hdb2 /mnt/hdb2
# assign today's date to xx, example 071130 on 30Nov2007
xx=`date +%y%m%d`
# make a directory in the folder "partimage" on hdb2 and name it for the date
mkdir /mnt/hdb2/partimage/$xx
cd /mnt/hdb2/partimage/$xx
# write start time to a logfile
zz=$xx'logfile'
echo 'start at: '`date`>$zz
# make an image of suse102_root options: -z1=gzip -d=no description save=save_image -b=batch(not gui) -f3=quit when finished
partimage -z1 -d save -b -f3 /dev/hda5 /mnt/hdb2/partimage/$xx/hda5.partimg.gz
# make an image of /home options: -z1=gzip -d=no description save=save_image -b=batch(not gui) -f3=quit when finished
partimage -z1 -d save -b -f3 /dev/hda6 /mnt/hdb2/partimage/$xx/hda6.partimg.gz
# write contents of file autorun to a file in the target directory
cat /mnt/cdrom/autorun >>script.used
# write end time to the logfile
echo 'end at: '`date`>>$zz
# write the contents of the backup directory into the logfile
ls -l>>$zz
reboot
 

These are the things to customise: Change hdb2 to match your target storage partition. Change hda5 to match your root partition. Change hda6 to match your home partition. Everything else should match your system.

This is the key line:

partimage -z1 -d save -b -f3 /dev/hda5 /mnt/hdb2/partimage/$xx/hda5.partimg.gz

If you have six partitions, duplicate this line six times, replacing hda5 with your correct partition designations and hdb2 with your target storage/backup partition.

That's all there is folks, enjoy.

Recommended Links

---

In case of broken links please try to use Google search. If you find the page please notify us about new location

CD and DVD Archiving: Quick Reference Guide for Care and Handling (NIST): http://www.itl.nist.gov/div895/carefordisc/disccare.html

• Magnetic Tape Storage and Handling: A Guide for Libraries and Archives (NML): http://www.imation.com/america/pdfs/AP_NMLdoc_magtape_S_H.pdf

• The Linux Administrator's Security Guide is a security guide with an excellent section on the practicalities of Linux backup and recovery.
   
• Introduction to Backing Up and Restoring Data is an overview that is independent of operating system or system architecture. In this article, the author explores backup techniques as well as planning backups.
   
• Linux Administration Made Easy is an older reference, but still useful, as the general procedures and techniques for Linux have remained consistent.
   
• The Linux System Administrator's Guide is an introduction to system administration of a Linux system for novices.
   
• The TAO of Backup is an interesting presentation of backup philosophy, presented as philosophy. It is associated with a commercial product, but the information is well written.
   
• The IBM developerWorks tutorial "Backing up your Linux machines" walks you through the process of creating and following through with a backup strategy.
   
• Storing backup data and other bits on a CD is easy: learn how in the IBM developerWorks article "Burning CDs on Linux".
   
• If you're transitioning to Linux from a Windows environment, you'll also want to read the Technical FAQ for Linux users.
• 
   
• File permissions and security are addressed in Chapter 3 of the Introduction to Linux guide at the Linux Documentation Project.