Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

Perl and Shell Backup Scripts

News

Simple Unix Backup Tools

Recommended Books

Recommended Links

rsnapshot

Etc directory backup

Backing Up And Restoring Linux With SystemImager

Baseliners Suse supportconfig Red Hat sosreport UNIX - Linux Copy Master Boot Record (MBR) SCC Dump and restore udev
Emergency Restore Procedure Loopback filesystem Labeling the partition Booting into Rescue Mode LVM Linux backup and recovery Linux Swap filesystem
Grub How To Manage Your Disk By UUID Data deduplication Linux filesystems Sysadmin Horror Stories Humor Etc

Introduction


Yesterday,
All those backups seemed a waste of pay.
Now my database has gone away.
Oh I believe in yesterday.

unknown source (well originally Paul McCartney :-)

One thing that distinguishes a professional Unix sysadmin from an amateur is attitude to backups and the level of knowledge of backup technologies. Professional sysadmin knows all too well that often the difference between major SNAFU and a nuisance is availability of the up-to-date backup of data.  Here is pretty telling poem (From unknown source (well, originally, by Paul McCartney :-) on the subject :

Yesterday,
All those backups seemed a waste of pay.
Now my database has gone away.
Oh I believe in yesterday.

You can read more about this important issue at Sysadmin Horror Stories

Unix and linux suffer from neglect: there are very few good back up solutions. Dump and restore are hopelessly outdated. Tar and cpio work with individual filesystems/partitions not with the disk as whole. 

Partimage was attempt to replicate Ghost functionality on Linux, but unfortunately is no longer supported. Rsnapshot  is an interesting idea that utilizes capabiltiesi of modern USB drives and not replicated tape base "mentality" of tools like tar (used in written in bash  Relax-and-Recover) . See also Bare metal recovery of Linux systems

Poor man implementation of partimage that supports filesystems like ext4 can be created using classic Unix tools and bootable CD with the same flavor of Linux (this is an important consideration, see below). You might also need  NFS support (to mount the image)

You can create full backup of your system the following way:

In this case restore process is as following

Please not that a bootable disks like SystemRescueCD or Knoppix are not very convenient/usable to with RHEL as they do not understand security attributes used in RHEL. USB drive created from the standard RHEL ISO is the only reliable rescue disk for this flavor of Linux

The three major backup programs are dump/restore, tar, and cpio. Their functionality can be easily enhanced by Perl scripts.

Backup and restore of MBR

Create a backup of your MBR by doing a:

dd if=/dev/hdx of=MBR-backup bs=512 count=1

To restore MBR use

dd if=MBR-backup of=/dev/hdx bs=512 count=1 

the same can be done with sfdisk  utility.

When you're multi-booting or installing a new operating system onto a used system, sometimes the MBR (Master Boot Record) gets all messed up, so you need to wipe it out and start over. You can do this with the dd command. Be sure to use your own drive name for the of= value:

# dd if=/dev/zero of=/dev/hda bs=446 count=1

That preserves the partition table. If you also want to zero out the partition table, do this:

# dd if=/dev/zero of=/dev/hda bs=512 count=1

See also  discusiion at Ubuntu Blog

Michael Manning - October 20, 2005
If you are going to need to use a Live CD to recover the backup MBR then you could have booted into rescue mode from the install CD (of any linux distro you choose) and mounted the linux partition, chrooted to the root partition and typed grub-install /dev/hdx.

mkdir /mnt/temp
mount /dev/hda3 /mnt/temp ## if hda3 was the linux partition
chroot /mnt/temp
grub-install /dev/hda

That would install the grub first stage boot loader to you the MBR of the hard disk and presented you with the same options you had before the Windows reinstall. no need for backup MBR records!

Just my two bobs worth – there is always more than one way to skin a cat.
Michael Manning

Reply
 2. ubuntonista - October 20, 2005
Thanks Michael!

Reply

 3. zenith - October 20, 2005
Very useful post, like so many here. Thanks for putting together a great Ubuntu blog.

Reply

 4. Jason - October 20, 2005
Note that the grub-install method has the added protection of not inadvertantly overwriting your partition table if it had been modified since your last MBR backup.

This is because the 512byte MBR sector is actually two parts:
1) The first 446 bytes is the grub stage1 bootloader (or the windows bootloader after you’ve reinstalled windows and it “helpfully” overwrites grub).
2) The last 64 bytes is where your partition table is stored.

So, if you only want to backup the bootloader in the MBR, remember to change the bs=512 to bs=446.

Reply
 5. kOoLiNuS - October 21, 2005
this is one of the “problems” of the single disk ubuntu … they actually don’t have a real rescue mode so you need a live-cd or another distro 1st cd …

they know it, since there are some posts on this subject dated some month ago in the archives …

 

Using Perl as a glue for existing Linux backup tools

Linux does not have the traditional UNIX backup programs dump and restore. Only tar and cpio are present. It is also possible to use dump and restore in a more secure fashion over ssh.

The GNU tar supports remote devices using the same syntax as rdump.

The same could be accomplished with  by using a pipeline and rsh to send the data to a remote tape drive.

# tar cf - . | rsh hostname dd of=tape-device obs=20b

If you are worried about the security of backing up over a network you should use the ssh command instead of rsh.

cpio  can backup and restore hard links and special files. This last feature makes cpio a better choice for filesystem backup. cpio does not know how to walk the directory tree and a list of files must be provided through stdin.

cpio does not support backups across the network. You can use a pipeline and ssh to send the data to a remote server

# for f in directory_list; do
find $f >> backup.list
done
# cpio -v -o --format=newc < backup.list | ssh user@host "cat > backup_device"

Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Aug 29, 2017] backup-etc.sh -- A script to backup the /etc directory

This is simple script that generated "dot" progression lines. Backup name includes a timestamp. No rotation implemented.
Aug 29, 2017 | wpollock.com
   #!/bin/bash
# Script to backup the /etc heirarchy
#
# Written 4/2002 by Wayne Pollock, Tampa Florida USA
#
#  $Id: backup-etc,v 1.6 2004/08/25 01:42:26 wpollock Exp $
#
# $Log: backup-etc,v $
# Revision 1.6  2004/08/25 01:42:26  wpollock
# Changed backup name to include the hostname and 4 digit years.
#
# Revision 1.5  2004/01/07 18:07:33  wpollock
# Fixed dots routine to count files first, then calculate files per dot.
#
# Revision 1.4  2003/04/03 08:10:12  wpollock
# Changed how the version number is obtained, so the file
# can be checked out normally.
#
# Revision 1.3  2003/04/03 08:01:25  wpollock
# Added ultra-fancy dots function for verbose mode.
#
# Revision 1.2  2003/04/01 15:03:33  wpollock
# Eliminated the use of find, and discovered that tar was working
# as intended all along!  (Each directory that find found was
# recursively backed-up, so for example /etc, then /etc/mail,
# caused /etc/mail/sendmail.mc to be backuped three times.)
#
# Revision 1.1  2003/03/23 18:57:29  wpollock
# Modified by Wayne Pollock:
#
# Discovered not all files were being backed up, so
# added "-print0 --force-local" to find and "--null -T -"
# to tar (eliminating xargs), to fix the problem when filenames
# contain metacharacters such as whitespace.
# Although this now seems to work, the current version of tar
# seems to have a bug causing it to backup every file two or
# three times when using these options!  This is still better
# than not backing up some files at all.)
#
# Changed the logger level from "warning" to "error".
#
# Added '-v, --verbose' options to display dots every 60 files,
# just to give feedback to a user.
#
# Added '-V, --version' and '-h, --help' options.
#
# Removed the lock file mechanism and backup file renaming
# (from foo to foo.1), in favor of just including a time-stamp
# of the form "yymmdd-hhmm" to the filename.
#
#

PATH=/bin:/usr/bin

# The backups should probably be stored in /var somplace:
REPOSITORY=/root
TIMESTAMP=$(date '+%Y%m%d-%H%M')
HOSTNAME=$(hostname)
FILE="$REPOSITORY/$HOSTNAME-etc-full-backup-$TIMESTAMP.tgz"

ERRMSGS=/tmp/backup-etc.$$
PROG=${0##*/}
VERSION=$(echo $Revision: 1.6 $ |awk '{print$2}')
VERBOSE=off

usage()
{  echo "This script creates a full backup of /etc via tar in $REPOSITORY."
   echo "Usage: $PROG [OPTIONS]"
   echo '  Options:'
   echo '    -v, --verbose   displays some feedback (dots) during backup'
   echo '    -h, --help      displays this message'
   echo '    -V, --version   display program version and author info'
   echo
}

dots()
{  MAX_DOTS=50
   NUM_FILES=`find /etc|wc -l`
   let 'FILES_PER_DOT = NUM_FILES / MAX_DOTS'
   bold=`tput smso`
   norm=`tput rmso`
   tput sc
   tput civis
   echo -n "$bold(00%)$norm"
   while read; do
      let "cnt = (cnt + 1) % FILES_PER_DOT"
      if [ "$cnt" -eq 0 ]
      then
         let '++num_dots'
         let 'percent = (100 * num_dots) / MAX_DOTS'
         [ "$percent" -gt "100" ] && percent=100
         tput rc
         printf "$bold(%02d%%)$norm" "$percent"
         tput smir
         echo -n "."
         tput rmir
      fi
   done
   tput cnorm
   echo
}

# Command line argument processing:
while [ $# -gt 0 ]
do
   case "$1" in
      -v|--verbose)  VERBOSE=on; ;;
      -h|--help)     usage; exit 0; ;;
      -V|--version)  echo -n "$PROG version $VERSION "
                     echo 'Written by Wayne Pollock '
                     exit 0; ;;
      *)             usage; exit 1; ;;
   esac
   shift
done

trap "rm -f $ERRMSGS" EXIT

cd /etc

# create backup, saving any error messages:
if [ "$VERBOSE" != "on" ]
then
    tar -cz --force-local -f $FILE . 2> $ERRMSGS 
else
    tar -czv --force-local -f $FILE . 2> $ERRMSGS | dots
fi

# Log any error messages produced:
if [ -s "$ERRMSGS" ]
then logger -p user.error -t $PROG "$(cat $ERRMSGS)"
else logger -t $PROG "Completed full backup of /etc"
fi

exit 0

Perl-based Backup Manager download

SourceForge.net

Command-line perl script with configuration files that allow you to mix and match backup tools to manage your monthly backups. You can for eg, configure a command to download files, zip them and copy it into another disk, all using different tools.

Perl-based Backup Manager Web Site

BackupPC

is a high-performance, enterprise-grade backup system for backing up Linux, Win32, and laptops to a server's disk. Features include clever pooling of identical files, no client-side software, and a powerful Apache/CGI user interface. It is written in Perl

ESR Backup

ESR Backup (Easy, Secure, Remote) is a backup system for UNIX and Linux that is easy to use and configure. It automatically performs full and incremental backups on files you choose, and automatically rotates and archives your backups, encrypts archives with a 448 bit key, and can store archives on a remote server via FTP or on the local file system.

Simplebackup

Simplebackup is a cross-platform backup program. It reads a configuration file, then it builds a compressed file for each of your backup directories or files on your backup list, and places the compressed files into another location. For example, this location can be a network mapped drive in Windows, an NFS mounted drive in Unix, another hard disk, an FTP server, an SFTP (Secure FTP) server, an HTTP (WebDAV) server, one or more email accounts, or a tape device (Unix only). This will duplicate your information, doing the so called "backup".

storebackup

Written in Perl

freshmeat.net

storebackup is a backup utility that stores files on other disks. It includes several optimizations that reduce the disk space needed and improve performance, and unifies the advantages of traditional full and incremental backups. Depending on its contents, every file is stored only once on disk. It includes tools for analyzing backup data and restoring. Once archived, files are accessible by mounting filesystems (locally, or via Samba or NFS). It is easy to install and configure.

Backup Manager

Backup Manager is a tool for generating archives easily. It is designed for those who don't want an obfuscated tool for backing up their system. It can make tar, tar.gz, tar.bz2, and zip archives, can perform incremental backups, and can upload archives to remote hosts with FTP, SSH, or RSYNC. It can be run in a parallel mode with different configuration files. The configuration file is simple to understand. The backup process can also be customized thanks to hook scripts.

flexbackup

flexbackup is a configurable and easy to use Perl-based backup tool, that can backup local files as well as remote machines (using ssh). It allows the backup itself to be made with afio, cpio, tar, dump, star, or pax. It can work with tape drives, on-disk archive files, or on-disk directory trees.

dobackup.pl

dobackup.pl is a flexible Perl script to handle unattended incremental backups of multiple servers. It handles multiple media sets with automatic media preparation and rotation, configurable 'what-to-backup', global per-host exclusion patterns, and user settable 'don't-back-this-up' metafiles. Its design goal is zero-maintenance, nothing to do except change the media when told.

fsbackup by Maxim Chirkov

Perl-based file system backup and synchronization utility.

fsbackup is an incremental backup creation utility. It supports backup compression and encryption. Backups can be stored on the local file system, and a remote host (via SSH, or FTP). Some additional scripts allow backup SQL tables from PostgreSQL and MySQL, save system configuration files, and a list of installed packages. Backed-up files can be recovered, and system packages can be reinstalled.

confstore

A configuration backup utility that makes backups of configuration.

Confstore is a configuration backup utility. It scans a system for all recognised configuration files and then stores them in a simple archive. It knows what to scan for by reading a definitions file. Confstore can also restore configuration from backup archives it has previously created.

NasBackup

NasBackup is a solution for backing up desktop PCs, laptops, and servers to network disks. It is highly configurable and uses rsync to only transfer file differences over the network.

Shell-based

backup2l

A lightweight tar-based backup/restore tool.

Bontmia

An incremental network backup tool for snapshotting directories.

Base Utilities

Partition Image is a Linux/UNIX utility similar to Symantec's Ghost. This utility saves partitions in the EXT2, Reiserfs, NTFS, HPFS, FAT16, and FAT32 file system formats to an image file. The image file can be compressed with gzip or bzip2 in order to save disk space, and it can be split in order to fit onto a series of floppy disks. This program can be useful for backup purposes. A boot/root disk is also provided, allowing you to run Partition Image without Linux installed on the hard disk.

Dump-Restore

The dump package contains both dump and restore. Dump examines files in a filesystem, determines which ones need to be backed up, and copies those files to a specified disk, tape or other storage medium. The restore command performs the inverse function of dump; it can restore a full backup of a filesystem. Subsequent incremental backups can then be layered on top of the full backup. Single files and directory subtrees may also be restored from full or partial backups.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)



Etc

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes.   If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner. 

ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.  

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least


Copyright © 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

Last modified: August 29, 2017