Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Cloning systems, disks and partitions

News

Recommended Books

Recommended Links Partimage Disk Backup  Cloning parition table on a new disk  Installing Suse from ISO image
Prepare a RHEL host for cloning dd GNU ddrescue   Admin Horror Stories Humor Etc

The traditional UNIX® backup programs are dump and restore. They operate on the drive as a collection of disk blocks, below the abstractions of files, links and directories that are created by the file systems. dump backs up an entire file system on a device. It is unable to backup only part of a file system or a directory tree that spans more than one file system. dump does not write files and directories to tape, but rather writes the raw data blocks that comprise files and directories.

Several programs can be used for a cloning a whole system on the disk

Following programs can be used to clone partitions:

In the days when the programs dump and restore were the preferred backup solution, many system administrators would take the filesystems offline (that is, unmount them) before backing up. There was a risk that dump would fail if a live filesystem were updated during the dump. This technique was possible because dump opened the raw device (such as /dev/hda1), and decoded the filesystem structure directly, rather than going through the usual system-call interface to access the directories and files. This technique won't work for tools like tar, which (like any other normal application) require a filesystem to be mounted before its contents can be accessed. There is a small risk that a file will end up corrupt on a tar archive if it happens to get updated while tar is in the middle of reading it. If this risk is not acceptable, one option is to bring the system down to single-user mode, so that the filesystems are quiescent.

If your filesystem is on a logical volume, another option is to take a snapshot of the volume, then archive the snapshot.You can create a new logical volume that contains a snapshot of this filesystem with the commands:

# modprobe dm-snapshot
# lvcreate 
 -L 512M -s -n ftpsnap /dev/system/ftp

Here, the modprobe command was necessary to load the device mapper snapshot module into the kernel (one of the few times I've found it necessary to load a module explicitly). The -s flag to lvcreate says to create a snapshot. The -n flag gives the snapshot logical volume a name (it will be created within the system volume group) and the final argument names the existing logical volume you want to take the snapshot of. After this, you have two logical volumes, like this:

# lvscan
  ACTIVE   Original '/dev/system/ftp' [512.00 MB] inherit
  ACTIVE   Snapshot '/dev/system/ftpsnap' [512.00 MB] inherit

Now you can mount the snapshot:

# mount /dev/system/ftpsnapshot /mnt

and use tar, for example, to safely archive the files from it.

Creating a snapshot logical volume does not just blindly copy the whole of the original volume into the snapshot—it's more intelligent than that. A snapshot contains only those blocks that differ from the original. As the original or the snapshot is written to, obviously the number of different blocks will grow.


Top updates

Bulletin Latest Past week Past month
Google Search


NEWS CONTENTS

Old News ;-)

[Dec 04, 2013] Backup-Recovery - Cloning Hard Drives with GNU-Linux

2002-12-25

It is quite easy to clone identical hard drives using the dd command on GNU/Linux. Make sure that you put the source drive and destination drive in the system so that they don't affect the boot. If you have a SCSI system, this is most likely done by making the SCSI IDs higher. With IDE, you probably need to put the drives in as secondaries on either channel, assuming your CD-ROM is the primary device on the second chain. On our system, we made the source ID 4 and the destination ID 6. Verify this using dmesg:

[root@srv-33 root]# dmesg | grep sd
Kernel command line: ro root=/dev/sda1 console=ttyS0 
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
Attached scsi disk sdb at scsi0, channel 0, id 4, lun 0
Attached scsi disk sdc at scsi0, channel 0, id 6, lun 0
SCSI device sda: 2069860 512-byte hdwr sectors (1060 MB)
sda: sda1 sda2
SCSI device sdb: 8388315 512-byte hdwr sectors (4295 MB)
sdb: sdb1 sdb2
SCSI device sdc: 8388315 512-byte hdwr sectors (4295 MB)
sdc: unknown partition table
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,1), internal journal
[root@srv-33 root]# 
Notice that with this copy method, we don't need to worry about partitions, boot sectors, etc. To copy sdb to sdc:
[root@srv-33 root]# dd if=/dev/sdb of=/dev/sdc &
[1] 612
[root@srv-33 root]# 
The ampersand throws the task in the background. You may then copy to other destination drives if you wish. Again, this copies everything on the drive (REALLY!!), so you can image any operating system. One cheapie way to recover a server is to image the hard drive in this way. The SIDs (Windows) will be identical as well. You will be at the exact same state as the time of imaging. No worries about open files, as long as you shut down the OS properly on the drive first. This method is not suitable for cloning Windows (NT and above) workstations that will be up at the same time, unless you feel comfortable changing the SIDs, etc. Test this first in a non-production environment, and, oh yeah, read our terms of use.

[Nov 21, 2008] rsnapshot - Local-Remote Filesystem backups utility in openSUSE SUSE & openSUSE

Probably does not make much sense but Perl codebase might be reused.

Posted by admin on October 4th, 2008

rsnapshot is a filesystem backup utility based on rsync. Using rsnapshot, it is possible to take snapshots of your filesystems at different points in time. Using hard links, rsnapshot creates the illusion of multiple full backups, while only taking up the space of one full backup plus differences. When coupled with ssh, it is possible to take snapshots of remote filesystems as well.

rsnapshot is written in Perl, and depends on rsync. OpenSSH, GNU cp, GNU du, and the BSD logger program are also recommended, but not required. rsnapshot is written with the lowest common denominator in mind. It only requires at minimum Perl 5.004 and rsync. As a result of this, it works on pretty much any UNIX-like system you care to throw at it.

rsnapshot can run almost out of the box with very little configuration changes although advanced configurations can be done with little more effort.

SystemRescueCD backup partitions and drives

HowTo: Install SystemRescueCD on a Dedicated Hard Disk Partition.
OR
HowTo: No-effort Backup Solution for Partitions and Hard Drives

Introduction: Downloading and burning SystemRescueCD provides a bootable Gentoo-based distro on a CD. The installed applications focus on restoring disabled Linux/windows distros on the hard drives, or retrieving data if things go terribly wrong. You can operate direct from the booted CD. It's great. But SystemRescueCD also contains PartImage, the powerful free Linux alternative to Norton Ghost. So it's a too-easy tool for backing up single or multiple partitions, or whole drives.

Here I recount HowTo install SystemRescueCD onto a dedicated partition. I include a script for creating backup images of your hard drive partitions. Once you go through this tutorial as a practical exercise, you'll have the knowledge and confidence to customise all manner of backup solutions so very easily.

This tutorial is for the middle ground reader, too hard for new Linux users and too simple for Gurus. It's drawn from material in the On Line Manual at the System Rescue CD Site.

Summary of the steps for installing SystemRescueCD on a dedicated hard disk partition:

  1. Prepare a separate SystemRescue partition
  2. Download the SystemRescueCD ISO file
  3. Extract bootable image files from the ISO to the boot partitiion
  4. Edit Suse's GRUB configuration to facilitate booting the SystemRescue partition
  5. Prepare and place your scripts, if any
  6. Boot with Suse's loader --> select item SystemRescueCd

Step 1: Prepare a separate SystemRescue partition: I leave it to you to make the partition. You need about 160Mb plus any extra storage you might need. I use 400Mb. Note that this partition becomes the root of the cdrom after SysRescueCD boots from it, so its filesystem becomes read-only. This means you will select writeable workspaces on other partitions.

Suppose for illustration that you have prepared partition hda13 for the installation. Now make a directory to mount hda13 into openSUSE, e.g. /SysRescCD. You can mount hda13 with this command:

mount /dev/hda13 /SysRescCD

BUT I use the more convenient permanent mount created by placing this line into /etc/fstab:

/dev/hda13 /SysRescCD ext3 defaults 1 2

You can do that with Yast --> System --> Partitioner OR more simply by issuing this command in a console: kdesu kwrite /etc/fstab and then typing the line in.

Step 2: Download the SystemRescueCD ISO file: You can download the CD ISO for the SystemRescueCD by following this project download link. The ISO filename looks like this: systemrescuecd-x86-x.y.z.iso. Place it anywhere on your hard drives, at e.g. /path_to/systemrescuecd-x86-x.y.z.iso

Step 3: Extract bootable image files from the ISO and place them in boot partition: You can mount the ISO file for viewing the files on the CD. First create a folder to mount the ISO in, e.g. /iso. Then mount the ISO with this command in a root terminal:

mount -o loop -t iso9660 /path_to/systemrescuecd-x86-0.3.6.iso /iso

You'll find these three files of special interest on these paths inside the mount folder:

/iso/sysrcd.dat
/iso/isolinux/rescuecd
/iso/isolinux/rescuecd.igz

Create the folder sysrcd in the root of hda13, using the mount point /SysRescCD to place it at /SysRescCD/sysrcd. Then copy the three files into folder sysrcd. The name sysrcd is immutable.

The partition hda13 is now configured with the bootable Gentoo distro and all that remains is to point Suse's bootloader at it.

Step 4: Edit Suse's GRUB configuration to facilitate booting the SystemRescue partition: You can open the Grub configuration file in a text editor with commands like this one for Kwrite:

kdesu kwrite /boot/grub/menu.lst

Edit/add these lines at the bottom of the file, one blank line below the last entry:

title SystemRescueCd
root (hd0,12)
kernel /sysrcd/rescuecd root=/dev/ram0 init=/linuxrc looptype=squashfs loop=/sysrcd/sysrcd.dat splash=silent nosound subdir=sysrcd cdroot=/dev/hda13 setkmap=us vga=0x31a
initrd /sysrcd/rescuecd.igz
boot

Remember to adapt my (hd0,12) which is for my hda13, across to your situation. Also, note that the sequence beginning "kernel" and ending "0x31a" is all the same/one line. I've included three parameters at the end: cdroot=/dev/hda13 setkmap=us vga=0x31a. These set the distro upto have hda13 at the root of the cd (on /mnt/cdrom), to have the US keyboard and for a vga screen that suits me. If you wanted to boot into the Window Manager Desktop Environment to access GUI tools, you would use this line instead:

kernel /sysrcd/rescuecd root=/dev/ram0 init=/linuxrc looptype=squashfs loop=/sysrcd/sysrcd.dat splash=silent nosound subdir=sysrcd cdroot=/dev/hda13 setkmap=us vga=0 dostartx

A list of boot options can be seen on this link at the SystemRescueCD site.

Step 5: Prepare and place your scripts, if any: Pre defined sites [like the floppy disk, the root of the CDROM, the root of the installation partition] are searched straight after booting for scripts which if found are executed. You can have one or many scripts. See the SystemRescueCD site for full details. I'll deal with only one location here: the root of the installation partition. It's really simple. Rust create a script called autorun and lodge it in the root of the installation partition, hda13. It will run just after the system boots to a console.

Step 6: Boot with Suse's loader: Reboot and select item SystemRescueCd. The root of partition hda13 automounts at location /mnt/cdrom in the booted-up virtual filesystem. All files and scripts placed on the partition are thus available at /mnt/cdrom.

Backup Script: I constantly change the filesystems on my primary hard drive and it's hard to prevent damage to them. So I back them up regularly. This takes a long time. I use a script called autorun in the root partition of hda13 and simply boot to SystemRescueCD on hda13 and walk away to let the job proceed. Here's my scenario and script. You could easily modify the script for your scenario.

Scenario: I have a Suse root partition at hda5 and a /home partition at hda6. These have to be backed up when they're not being used, i.e. from within another operating system. The Gentoo installation on the SystemRescueCD partition contains a script, "autorun", which employs "partimage", the Linux free version of "Ghost". It is perfect for the task. I have prepared a folder called "partimage" on partition hdb2 on IDE2 drive. The script mounts hdb2 into Gentoo/SystemRescueCD's filesystem, generates a date-coded folder on hdb2 and copies image files across from the Suse root and home partitions.

Script

#!/bin/sh
# mount the target directory
mkdir /mnt/hdb2
mount /dev/hdb2 /mnt/hdb2
# assign today's date to xx, example 071130 on 30Nov2007
xx=`date +%y%m%d`
# make a directory in the folder "partimage" on hdb2 and name it for the date
mkdir /mnt/hdb2/partimage/$xx
cd /mnt/hdb2/partimage/$xx
# write start time to a logfile
zz=$xx'logfile'
echo 'start at: '`date`>$zz
# make an image of suse102_root options: -z1=gzip -d=no description save=save_image -b=batch(not gui) -f3=quit when finished
partimage -z1 -d save -b -f3 /dev/hda5 /mnt/hdb2/partimage/$xx/hda5.partimg.gz
# make an image of /home options: -z1=gzip -d=no description save=save_image -b=batch(not gui) -f3=quit when finished
partimage -z1 -d save -b -f3 /dev/hda6 /mnt/hdb2/partimage/$xx/hda6.partimg.gz
# write contents of file autorun to a file in the target directory
cat /mnt/cdrom/autorun >>script.used
# write end time to the logfile
echo 'end at: '`date`>>$zz
# write the contents of the backup directory into the logfile
ls -l>>$zz
reboot

These are the things to customise: Change hdb2 to match your target storage partition. Change hda5 to match your root partition. Change hda6 to match your home partition. Everything else should match your system.

This is the key line:

partimage -z1 -d save -b -f3 /dev/hda5 /mnt/hdb2/partimage/$xx/hda5.partimg.gz

If you have six partitions, duplicate this line six times, replacing hda5 with your correct partition designations and hdb2 with your target storage/backup partition.

That's all there is folks, enjoy.

Recommended Links

Google matched content

Softpanorama Recommended

CD and DVD Archiving: Quick Reference Guide for Care and Handling (NIST): http://www.itl.nist.gov/div895/carefordisc/disccare.html



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: October 20, 2015