|Home||Switchboard||Unix Administration||Red Hat||TCP/IP Networks||Neoliberalism||Toxic Managers|
May the source be with you, but remember the KISS principle ;-)
Bigger doesn't imply better. Bigger often is a sign of obesity, of lost control, of overcomplexity, of cancerous cells
|News||Recommended Links||Working with Images||Disk Cloning||File tree synchronization||Disk Repartitioning||Rar|
|Working with dd Images||Working with ISO Images||dd||Ghost||Alternatives to Norton Ghost|
|Disk Recovery||Fighting Spyware||Hard drives Failures||Admin Horror Stories||Linux Tips||Humor|
|Who is General Failure and why is he reading my disk ?
|If history repeats itself, and the unexpected always happens,
how incapable must Man be of learning from experience.
|"Experience keeps a dear school, but fools will learn in no other"
Nothing can increase interest in this topic more than a lost disk drive with data that does not have the most recent backup. Even if backup is just one week old, one realizes that the important data you was working on for several hours (may you best hours of the week) and everything that you had wrote is now gone forever. That happened to me recently when my old Seagate drive just stopped rotating exactly a week after the backup and I did not have any daily backups at all. As Benjamin Franklin aptly noted "Experience keeps a dear school, but fools will learn in no other."
You better take some steps to prevent this from happening again. Ever. Money are not a big question here as the cost of lost data often exceed price of good arching programs 100 times or more. You are not necessary locked in free/open source tools and you to find the best utility suitable for you. This is not an easy task as much depends on your style of working.
In the simplest case all source files and documents are usually kept on small partition (4G usually suffice for most people). This partition can be copied to safe storage using three types of intervals (dally, weekly and monthly). Each period can use unique method, for example folder sync for dally, partition on the second drive for weekly and backup or DVD for monthly backups.
Do not expect harddrive last long: for laptops anybody who is using a drive that more then three years old is taking
huge chances. If you data are valuable you better replace it with a new drive. See
Slashdot Google Releases Paper on
Disk Reliability. See also SUNRISE
drive statistics - the article is in Russian, but drive related diagram are self-explanatory.
If you data are valuable and you keep them on a laptop it make sense proactively replace the harddrive each three years. That actually makes laptop leases more attractive then they look otherwise :-)
Also all drive manufacturers have good years and bad years. Still you need to monitor drive statistics and with first SMART report take appropriate measures:
Our results confirm the findings of previous smaller population studies that suggest that some of the SMART parameters are well-correlated with higher failure probabilities. We find, for example, that after their first scan error, drives are 39 times more likely to fail within 60 days than drives with no such errors. First errors in reallocations, offline reallocations, and probational counts are also strongly correlated to higher failure probabilities. Despite those strong correlations, we find that failure prediction models based on SMART parameters alone are likely to be severely limited in their prediction accuracy, given that a large fraction of our failed drives have shown no SMART error signals whatsoever. This result suggests that SMART models are more useful in predicting trends for large aggregate populations than for individual components. It also suggests that powerful predictive models need to make use of signals beyond those provided by SMART."
If we try to classify available methods we can come to the following raw but still usable classification:
Image-based backup provides a mechanism that allows you to more completely recover a crashed system without having to spend time dealing with partitions, disk geometries, drive letter assignments, or drive formatting. The classic Unix utility for this purpose is dd. An example operation using dd could be as simple as:
dd if=/dev/rdsk/c0t3d0s0 of=/dev/rst0 bs=10k
for a straight image backup to the /dev/rst0 tape drive, or as elaborate as:
dd if=/dev/hda2 bs=10k count=1000 |gzip |tee /dev/rst0 \ |sum >/etc/images/dd_image`date +%b`.sum
To create the backup out of raw image you need to compress it with bzip2, rar or similar high performance archiver.
Old versions of Ghost were DOS programs and required that you reboot your system into DOS using their respective boot diskettes to perform the actual backup operation. This creates an inconvenience as the shutdown for a server or workstation is slightly disruptive, you are assured that the image produced by these tools will be complete and stable. Newer version (Ghost 9 and 10) can do it in Windows.
Since image-based backup only copies raw data, backups made using this method do not contain a direct catalog of files that can be used for the restoration of a single file. But that does not mean that restoring am image an all or nothing operation. Advanced utilities like Norton Ghost permit to mount a partition image as a logical drive and can recover a single file from within the image-based backup.
Also during restore Norton Ghost automatically adjust the restore process for disk parameters of the target disk, so within some reasonable range you can use a different disk for a target than you used for the backup. This feature is very convenient for disk upgrades. for example if you changed 20G drive to 40G drive on your laptop GHOST proportionally increases each partition so that they fill entire 40G. In this case you usually do not have any problems with installed applications.
Differences covered by Ghost and similar programs include a difference in the size, number of heads, number of tracks per cylinder, or even the numbers or size of the sectors in a given track.
This script tries to create ISO files of specific size from files/directories given at the command prompt. It relies heavily on mkisofs to calculate the ISO sizes and also for creation of the ISO files. The script will however include as many files as possible without changing the order of the files. If a small media size is selected, it may have to cut images short if the next file to be included is very large. Also, if a file results in a too large ISO file, a warning is printed and the image is created anyway.
Both Joliet and Rock Ridge options are passes to mkisofs which can result in some files unable to be included in the image. backup will detect the error and halt. You may have to work around that problem yourself. Also be warned that mkisofs will be run numerous times to determine exactly how many files can be put into the image (successive approximation technique is used).
Download the script.
Options:Usage: ./backup.sh [OPTION]... [PATH]... Create ISO files containing the directories to backup. Options: -b=# Max block size for ISO files (1 block=2048 bytes). -d Don't make ISO's -k Keep keep temporary files when done. -p=PREFIX filenames are PREFIX-1.iso,... (default: backup) -q Be less verbose. -s=# Max ISO size in bytes. -v Be more verbose. -cd Select 700MB CD media size. -dvd Select 4700000000 bytes DVD media size (default).
- SpinRite Disk Error Problem Detection (Score:2, Interesting) http://www.grc.com/sr/spinrite.htm [grc.com]). It claims to detect and repair disk errors before they are a problem with a low level scan. I bought it an used it on a server drive that had errors disk DOS file copies. It fixed the problem and no data was lost, but I don't have any other experience with it.
The program sounds pretty amazing from their web site.
Are many companies using it for preventative maintenance to avoid data loss on their servers?
- Re:SpinRite Disk Error Problem Detection Dissenting opinion. [grcsucks.com] The whole site [grcsucks.com] is an interesting read.
- One of TWO best papers at FAST (Score:3, Informative) This Google paper [usenix.org] just appeared at the 5th USENIX Conference on File and Storage Technologies [usenix.org] (a.k.a. FAST), the premier conference on file systems and storage. It won one of the best paper awards.
You might be interested in the other best paper award winner (in the shameless self-promotion department): TFS: A Transparent File System for Contributory Storage [usenix.org], by Jim Cipar [umass.edu], Mark Corner [umass.edu], and Emery Berger [slashdot.org] (Dept. of Computer Science, University of Massachusetts Amherst [umass.edu]). Briefly, it describes how you can make all the empty space on your disk available for others to use, without affecting your own use of the disk (no performance impact, and you can still use the space if you need it).
Dept. of Computer Science
University of Massachusetts Amherst
Backup only the necessary server disk sector contents
User-defined compression levels
Reduce your disk backup time and storage by excluding paging and hibernate files from the disk backup image
Manage a PC performance by changing the disk imaging process priority
Supports hard disks of all sizes
Create full images (everything on your PC), incremental images (changes since last backup), and differential images (changes since last full backup)
Use your PC during image creation with our no reboot feature
Verify disk backup image before a restore
Change partition type, file system, size, and disk location during restore
Check the file system after a restore
Acronis Secure Zone
Acronis Startup Recovery Manager
DriveImage XML V1.00
Backup and image logical drives and partitions, create hot images, copy one drive to another...
File Size: 1.5 MB
XXCOPY is simply a logical extension to XCOPY. It remains faithfully compatible with XCOPY in the invocation syntax, yet, adds many innovative features to be a very serious utility for anyone who feels comfortable in managing files in command line mode (DOS Box). XXCOPY has grown to be not just a file copy program but also file removal, search and list utility. Its short-name preserving capability makes it ideal to clone a system disk which can be made bootable. XXCOPY runs under Windows 95/98/ME/NT/2K/XP and comes with a 16-bit version XXCOPY16 which allows copying files in DOS environment using short name only (to be later expanded to long name). You may also use it for synchronizing systems. In short, it is an industrial strength utility for system administrators.
This program is great for recovering files from scratched CD's or defective floppy/hard disks.
Normally when your computer is unable to copy a file from a damaged disk it will abort and delete the portition of the file it has copied.
Unstoppable Copier will continue copying the file right to the end; any unrecoverable data after many retries is replaced with blanks. This will allow you to recover every byte of information that is available for recovery.
The program allows you to specify a single file, a group using wildcards (* and ?) or if you wish, select a starting folder and the program will copy all data from it and any sub folders it may contain.
This program doesn't just need to be used for copying files form defective disks, it can be used to transfer any files!
[Nov 28, 2005] Nero disk backup sucks. I used Nero disk backup for my weekly backups but never tested it. When my harddrive failed I discover the restore is a DOS utility and you need to map DVD drive in DOS or copy it to harddrive Fat32 partition to make it work. If there is a driver for DOS for your DVD drive you are file, but it there is no such driver your only option is to install additional IDE disk (or use existing FAT32 partition with enough space on it). Then you can copy the content on this disk and try your luck.
[Sept 11, 2005] Amazon.com Norton Ghost 9.0 - Disk Imaging Solution Software
Finally!! Somebody got it right., September 7, 2004
Reviewer: P. Clark (Los Angeles) - See all my reviews
After experimenting with Ghost version 9.0, I can report that Symantec has produced a slick piece of software now that PowerQuest's DriveImage capabilities have been fully integrated into the feature set. Note that Version 9.0 works only on XP and 2000 systems.
For my home machine, I'm using a spare IDE drive to store the backup images. I have set up a regular schedule which will automatically do a full baseline backup once per month, with weekly incremental saves. The backup is performed at the rate of about 1 GB per minute on an Intel 3.20 GHz machine, with compression set to the standard parameter of 40%. I set a limit of 3 baseline images so that the backup drive will never run out of room.
Backup jobs are run within Windows so there is no need to boot to DOS. Backups can also be transmitted to a network server or written directly to CD or DVD devices.
Ghost includes a nifty utility which can open a backup image so that you can pull out a specific file for restoration. Following a catastrophic failure, you can boot from the Ghost software CD into recovery mode and very easily restore an entire drive including the MBR, making the disk ready to boot.
Ghost will also copy a drive while running under Windows. I successfully copied my C-drive, but XP would not boot the copy without permission from Microsoft.
Backups using removable media (i.e. CD or DVD) must be started manually. Backups to a hard drive or a network server can be scheduled for automatic operation. Just leave the PC on at night and it's all taken care of. Ghost will even send notification via email that the backup was completed.
By using Backup you can create a duplicate copy of all hard disk and then archive it on another storage device, such as a hard disk or a tape.
You can easily restore it by using the Restore or Automated System Recovery Wizards.
To start Backup or to access Restore and Automated System Recovery
• Click Start, click All Programs, click Accessories, click System Tools, and then click Backup.
Windows XP Backup, Restore, and Automated System Recovery all function when Windows XP Professional is functioning.
If your computer does not start properly, you may need to use Recovery Console. Recovery Console provides a command line during Startup from which you can make system changes when Windows XP Professional doesn't start.
To learn more about Backup, Restore, and Automated System Recovery, see Help and Support Center.
Google matched content
Unison (file synchronizer) - Wikipedia, the free encyclopedia
Slashdot Google Releases Paper on Disk Reliability
SystemRescueCd is a Linux system available from a bootable CDROM that provides an easy way to perform administrative tasks on your computer, such as creating and editing the partitions of the hard disk or backing up data. It contains a lot of system utilities (such as parted, partimage, and fstools), and basic programs (such as editors, midnight commander, and network tools). It also includes QtParted, a Partition Magic clone that makes editing partitons easy with its Qt graphical user interface. This CDROM aims to be very easy to use and accessible to everybody.
François Dupoux [contact developer]
GNU Parted (gparted) allows you to create, destroy, resize, and copy partitions. Supported partition types include ext2, FAT (FAT16 and FAT32), and Reiserfs filesystems and Linux swap devices. Supported disk labels include MS-DOS and PC98 partition tables, Sun and BSD disk labels, Macintosh partition maps, and raw access. Parted is useful for creating space for new operating systems, reorganising disk usage, copying data between hard disks, and disk imaging.
The mkCDrec utilities are optional for mkCDrec itself, but are an added value for rescue and recovery purposes. The utilities are staticly compiled and include parted, memtest, partimage, gpart, and recover. Memtest86 is also available for memory testing.
Partition Image Unlike Ghost you can same images to an external usb drive.
Partition Image is a Linux/UNIX utility similar to Symantec's Ghost. This uility saves partitions in the EXT2, Reiserfs, NTFS, HPFS, FAT16, and FAT32 file system formats to an image file. The image file can be compressed with gzip or bzip2 in order to save disk space, and it can be split in order to fit onto a series of floppy disks. This program can be useful for backup purposes. A boot/root disk is also provided, allowing you to run Partition Image without Linux installed on the hard disk.
by scribbler - Jul 10th 2003 11:23:59
Partition Image is a fantastic free alternative to the commercial Norton Ghost. Using the KNOPPIX live ISO, I was able to fully back up my Windows hard drive (FAT32), purposely destroy it using deltree, and fully restore it, without a single hitch from the program itself.
If I had to find a fault with the software, I'd have to say it's a little slow backing up/restoring. But that, to be honest is probably not a fault with the program. After all, it was a 2Gb FAT32 partition, and an hour either way to backup/restore the compressed data is a lot easier than having to install Windoze from scratch. The built-in functionality to compress the backup files works like a charm, it compressed those 2Gb's down to about 1.3Gb, easily enough space to fit onto CD.
It took an hour to back up a 10Gb ReiserFS partition on another machine without any compression. And considering the age/speed of the machine, this is totally adequate.
Overall, a fantastic program, and one I'm going to use a lot.
Backups -- In a League of Their Own Some historical data
Mondo is a free CD- or tape-based disaster-recovery suite for Linux and Windows. Supports LVM, RAID, almost any type of filesystem, bare metal restore. GPL.
Amanda is a free backup system designed to archive many computers on a network to a large-capacity tape drive. Amanda has many features and is widely used.
Bacula is a network client/server based backup program. Bacula is relatively easy to use and efficient, writes to multiple volumes, has many advanced storage management features that make it easy to find and recover lost or damaged files. Runs on Linux, Solaris, FreeBSD, with clients also on Windows, Irix and Mac OS X. GPL software.
Free windows 2000 backup downloads
Free Backup Software
Where can I get free backup software, Part 2
My Own Backup (MOB) A flexible piece of software that allows for Disk Spanning (CD-R backups). Some features include DOS command line support, Backup History and an Internet Live Update function.
Backup, Archival, and Synchronization Kit (BASK) BASK is a synchronization based backup and restore utility. It allows you to specify a source and destination, along with a mapping, a pattern to include, and a pattern and folders to exclude. You can determine file set differences through the date, time, and size of the source an tp://www.freebyte.com/fbbackup/">Free Byte Backup Allows one to easily copy (and filter) a large number of files and directories from various sources into one backup directory. It is possible to backup all files found in the specified set of input directories, or to have only certain file types copied. Files can be filtered according to file-extensions.
Back Up and Recover Your Information
About.com 2000 Backup Utility Basics
Chapter 5 Windows backup and restore
How do I clone-backup a hard disk under Windows
1..2..Freeware - Backup and Copy Utilities
The most popular utility in this class is Ghost (now owned by Symantec). See also Alternatives to Norton Ghost
Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy
War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes
Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history
The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor
The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D
Copyright © 1996-2018 by Dr. Nikolai Bezroukov. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) in the author free time and without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.
FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...
|You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info|
The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.
Last modified: July 07, 2013