Softpanorama May the source be with you, but remember the KISS principle ;-)
	Contents	Bulletin	Scripting in shell and Perl	Network troubleshooting	History	Humor

DD Command

The dd command has been around since the 1970s, ported to many systems, rewritten many times, and proved to be an indispensable Unix tool. The name is an allusion to IBM/360 mainframe OS JCL DD statement. It is jokingly said that dd stands for "destroy disk" or "delete data", since, being used for low-level operations on hard disks, a small mistake, such as reversing the if and of parameters, may accidentally render the entire disk unusable. In modern Linuxes you can mount dd image using loopback interface.

It served as an inspiration to the most important recent class backup programs called ghosters in memory of the original for Windows Ghost. The key idea is to get an image of the partition in the form of the file. DD uses older approach and includes into image parts of the disk that are not parts of the filesystem. That feature can be used to recovery of deleted files and in computer forensics when the contents of a disk need to be preserved as a byte-exact copy. In the latter case using cp command would not be sufficient because data from deleted files still physically present on a disk but are not visible in the file system naming space.

In Windows dd command can do backups and restores to the same disk but there are some problems with cloning Windows systems. Ghost is better tool for that. As a backup tool dd is still useful, but Ghost and its alternatives are much faster because they understand what part of the disk belongs to the filesystem and which don't. DD copies the whole partition or the whole disk indiscriminately.

But this weakness at the same time represents dd advantages over Ghost and similar, more modern, tools. In case of damaged filesystem dd is much more useful. You just need to create a DD-image and then search it for files using grep or similar tool. In most cases you can recover files pretty reliably using this method. I one manage to recover a script that was deleted several hours ago before it was discovered and server continued working all this time. That was the only copy and the user who did it has no backup. See Recovery of lost files using DD

On Linux Partimage is especially useful alternative to dd. Partition Image is a Linux utility which saves partitions for the most common filesystems formats (Ext3, NTFS, FAT32, etc) to a Ghost-style image file. The image file can be compressed in the GZIP/BZIP2 formats to save disk space, and split into multiple files. Partitions can be saved across the network since version 0.6.0.When using Partimage, the partitions must be unmounted.

The GNU clone of dd is part of fileutils package and was written by Paul Rubin, David MacKenzie, and Stuart Kemp. dd is also available for Windows as part of Microsoft Unix toolkit (SFU 3.5). It is also part of Cygwin. There is also native Windows Win32 port of dd (see Native Win32 ports of some GNU utilities and dd for windows)

Unlike most Unix commands, dd uses a keyword=value format for its parameters. This was modeled after IBM System/360 JCL, which had an elaborate DD 'Dataset Definition' specification for I/O devices in JCL language.

The following exit values are returned:

• 0 The input file was copied successfully.
• >0 An error occurred.

After completion, dd reports the number of whole and partial input and output blocks

A complete listing of all keywords is available via dd --help. It depends on flavor of dd used: GNU dd is different from, say, Solaris dd. See Reference

Basic dd Options

The basic syntax of dd is as follows:

# dd if= device of= device  bs= blocksize 

The preceding options are used almost every time you run dd:

1. if= file Specifies the input path. Standard input is the default. The if= argument specifies the input file or the file from which it is going to copy the data. This is the file or raw partition that you are going to back up (e.g., dd if=/dev/dsk/c0t0d0s0 or dd if=/home/file). If you want dd to look at stdin for its data, you don't need this argument.
   
2. of= file Specifies the output path. Standard output is the default. If the seek=expr conversion is not also specified, the output file will be truncated before the copy begins, unless conv=notrunc is specified. If seek=expr is specified, but conv=notrunc is not, the effect of the copy will be to preserve the blocks in the output file over which dd seeks, but no other portion of the output file will be preserved. (If the size of the seek plus the size of the input file is less than the previous size of the output file, the output file is shortened by the copy.) The output file The of= argument specifies the output file or the file to which you are sending the data. This could be a file on disk or an optical platter, another raw partition (e.g., dd of=/backup/file, dd of=/dev/rmt/0n). If you are sending to stdout, you don't need this argument. the block size

   The bs= argument specifies the block size, or the amount of data that is to be transferred in one I/O operation. This value normally is expressed in bytes, but in most versions dd also can be specified in kilobytes by adding a k at the end of the number (e.g., 10 K). (This is different from a blocking factor, like dump and tar use, which is multiplied by a fixed value known as the minimum block size. A blocking factor of 20 with a minimum block size of 512 would give you an actual block size of 10,240, or 10 K.) It should be noted that when reading from or writing to a pipe, dd defaults to a block size of 1. Generally on modern hardware bs=4096 is a reasonable minimum value. One cylinder (255 heards * 63 sectors * 512 = 16065. So bs=16065 also is attractive. Two cylinders fro multiplate disks (bs=32130) also might make sense.

   Changing block size does not affect how the data is physically written to a disk device, such as a file on disk or optical platter. Using a large block size just makes the data transfer more efficient. When writing to a tape device, however, each block becomes a record, and each record is separated by an interrecord gap. Once a tape is written with a certain block size, it must be read with that block size or a multiple of that block size. (For example, if a tape were written with a block size of 1024, you must use the block size of 1024 when reading it, or you may use 2048 or 10,240, which are multiples of 1024.) Again, this applies only to tape devices, not disk-like devices.

3. bs= n Sets both input and output block sizes to n bytes, superseding ibs= and obs=. If no conversion other than sync, noerror, and notrunc is specified, each input block is copied to the output as a single block without aggregating short blocks. This option is used only if ASCII or EBCDIC conversion is specified. For the ascii and asciib operands, the input is handled as described for the unblock operand except that characters are converted to ASCII before the trailing SPACE characters are deleted. For the ebcdic, ebcdicb, ibm, and ibmb operands, the input is handled as described for the block operand except that the characters are converted to EBCDIC or IBM EBCDIC after the trailing SPACE characters are added. Specifying the input and output block sizes separately When specifying block size with the option bs=, you are specifying both the incoming and outgoing block size. There are situations in which you may need a different block size on each. This is done with the ibs= and obs= options.
4. the number of records to read The count=n option tells dd how many records (blocks) to read. You can use this to read the first few blocks of a file or tape to see what kind of data it is, for example (see the following section). You can also use it to have dd tell you what block size a tape was written in (see below).

The dd utility copies the specified input file to the specified output with possible conversions. If you reverse the source and target, you can wipe out you file, partitions or even the whole disk. This feature has inspired the nickname "dd" Data Destroyer.

The standard input and output are used by default. The input and output block sizes may be specified for tape or just to increase efficiency (large blocks are generally transferred faster). Sizes are specified in bytes; a number may end with k, b, or w to specify multiplication by 1024, 512, or 2, respectively. Numbers may also be separated by x to indicate multiplication.

The dd utility reads the input one block at a time, using the specified input block size. dd then processes the block of data actually returned, which could be smaller than the requested block size. dd applies any conversions that have been specified and writes the resulting data to the output in blocks of the specified output block size.

Additional options

cbs= n

Specifies the conversion block size for block and unblock in bytes by n (default is 0). If cbs= is omitted or given a value of 0, using block or unblock produces unspecified results. bs is used only if ascii, asciib, unblock, ebcdic, ebcdicb, ibm, ibmb, or block conversion is specified. In the first two cases, cbs characters are copied into the conversion buffer, any specified character mapping is done, trailing blanks are trimmed, and a NEWLINE is added before sending the line to output. In the last three cases, characters up to NEWLINE are read into the conversion buffer and blanks are added to make up an output record of size cbs. ASCII files are presumed to contain NEWLINE characters. If cbs is unspecified or 0, the ascii, asciib, ebcdic, ebcdicb, ibm, and ibmb options convert the character set without changing the input file's block structure. The unblock and block options become a simple file copy.

files= n

Copies and concatenates n input files before terminating (makes sense only where input is a magnetic tape or similar device).

skip= n

Skips n input blocks (using the specified input block size) before starting to copy. On seekable files, the implementation reads the blocks or seeks past them. On non-seekable files, the blocks are read and the data is discarded.

iseek= n

Seeks n blocks from beginning of input file before copying (appropriate for disk files, where skip can be incredibly slow).

oseek= n

Seeks n blocks from beginning of output file before copying.

seek= n

Skips n blocks (using the specified output block size) from beginning of output file before copying. On non-seekable files, existing blocks are read and space from the current end-of-file to the specified offset, if any, is filled with null bytes. On seekable files, the implementation seeks to the specified offset or reads the blocks as described for non-seekable files.

count= n

Copies only n input blocks.

conv= value[,value. . . ]

Where values are comma-separated symbols from the following list:

ascii

Converts EBCDIC to ASCII.

asciib

Converts EBCDIC to ASCII using BSD-compatible character translations.

ebcdic

Converts ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand.

ebcdicb

Converts ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand.

ibm

Slightly different map of ASCII to EBCDIC. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand.

ibmb

Slightly different map of ASCII to EBCDIC using BSD-compatible character translations. If converting fixed-length ASCII records without NEWLINEs, sets up a pipeline with dd conv=unblock beforehand.

The ascii (or asciib), ebcdic (or ebcdicb), and ibm (or ibmb) values are mutually exclusive.

block

Treats the input as a sequence of NEWLINE-terminated or EOF-terminated variable-length records independent of the input block boundaries. Each record is converted to a record with a fixed length specified by the conversion block size. Any NEWLINE character is removed from the input line. SPACE characters are appended to lines that are shorter than their conversion block size to fill the block. Lines that are longer than the conversion block size are truncated to the largest number of characters that will fit into that size. The number of truncated lines is reported.

unblock

Converts fixed-length records to variable length. Reads a number of bytes equal to the conversion block size (or the number of bytes remaining in the input, if less than the conversion block size), delete all trailing SPACE characters, and append a NEWLINE character.

The block and unblock values are mutually exclusive.

lcase

Maps upper-case characters specified by the LC_CTYPE keyword tolower to the corresponding lower-case character. Characters for which no mapping is specified are not modified by this conversion.

ucase

Maps lower-case characters specified by the LC_CTYPE keyword toupper to the corresponding upper-case character. Characters for which no mapping is specified are not modified by this conversion.

The lcase and ucase symbols are mutually exclusive.

swab

Swaps every pair of input bytes. If the current input record is an odd number of bytes, the last byte in the input record is ignored.

noerror

Does not stop processing on an input error. When an input error occurs, a diagnostic message is written on standard error, followed by the current input and output block counts in the same format as used at completion. If the sync conversion is specified, the missing input is replaced with null bytes and processed normally. Otherwise, the input block will be omitted from the output.

notrunc

Does not truncate the output file. Preserves blocks in the output file not explicitly written by this invocation of dd. (See also the preceding of=file operand.)

sync

Pads every input block to the size of the ibs= buffer, appending null bytes. (If either block or unblock is also specified, appends SPACE characters, rather than null bytes.)

If operands other than conv= are specified more than once, the last specified operand=value is used.

For the bs=, cbs=, ibs=, and obs= operands, the application must supply an expression specifying a size in bytes. The expression, expr, can be:

1. a positive decimal number
2. a positive decimal number followed by k, specifying multiplication by 1024
3. a positive decimal number followed by b, specifying multiplication by 512
4. two or more positive decimal numbers (with or without k or b) separated by x, specifying the product of the indicated values.

All of the operands will be processed before any input is read.

Although you may think of dd as a bit copier, it also can manipulate the format of the data, such as converting between different character sets, upper- and lowercase, and fixed-length and variable-length records.

conv=ascii

Converts EBCDIC to ASCII

conv=ebcdic

Converts ASCII to EBCDIC

conv=ibm

Converts ASCII to EBCDIC using the IBM conversion table

conv=lcase

Maps US ASCII alphabetic characters to their lowercase counterparts

conv=ucase

Maps US ASCII alphabetic characters to their uppercase counterparts

conv=swab

Swaps every pair of bytes; can be used to read a volume written in different byte order

conv=noerror

Does not stop processing on an error

conv=sync

Pads every input block to input block size (ibs)

conv=notrunc

Does not truncate existing file on output

conv=block

Converts input record to a fixed length specified by cbs

conv=unblock

Converts fixed-length records to variable length

conv=..., ...

Uses multiple conversion methods separated by commas

Examples

• To make ISO from DVD :

  dd if=/dev/dvd of=dvd.iso # for dvd 

• Backing up your Master Boot Record (MBR).

  You should do this before you edit your partition table so that you can put it back if you mess things up.

  # dd if=/dev/hda of=/root/hda.boot.mbr bs=512 count=1

  If things mess up, you can boot with Knoppix, mount the partition containing /root (hda1 in this example) and put back the MBR with the command:

  # dd if=/mnt/hda1/root/hda.boot.mbr of=/dev/hda bs=512 count=1

  Note: You can backup only the MBR and exclude the partition table with the command:

   # dd if=/dev/hda of=/root/hda.mbr.noparttab bs=446 count=1

• Creating a hard drive backup directly to another hard drive (Dd - LQWiki)

  # dd if=/dev/hda of=/dev/sda conv=noerror,sync bs=4k

  This command is used often to create a backup of a drive (/dev/hda) directly to another hard drive (/dev/sda). (The device name /dev/hda is typical of an IDE hard drive, the device /dev/sda is typical of a USB disk.) This works only if the hard drive has enough storage to accommodate the source drive's filesystem. The advantage of this is that you do not have to mount the hard drive to make a backup and the only reference to hda is in /dev and in the command which is usually in a script in cron.

  The option "bs=4k" is used to specify the block size used in the copy. The default for the dd command is 512 bytes: use of this small block size can result in significantly slower copying. However, the tradeoff with larger block sizes is that when an error is encountered, the remainder of the block is filled with zero-bytes. So if you increase your block size when copying a failing device, you'll lose more data but also spend less time trying to read broken sectors. Tools like dd_rescue and dd_rhelp can provide a more flexible solution in such cases, combining the speed of a large block size for the regions without errors with finer-grained block-copies for regions with errors.

• Creating a compressed hard drive backup image

  # dd if=/dev/hda | gzip > /mnt/hdb1/system_drive_backup.img.gz

  Here dd is making an image of the first harddrive, and piping it through the gzip compression program. The compressed image is then placed in a file on a seperate drive. To reverse the process:

  # gzip -dc /mnt/hdb1/system_drive_backup.img.gz | dd of=/dev/hda

  Here, gzip is decompressing (the -d switch) the file, sending the results to stdout (the -c switch), which are piped to dd, and then written to /dev/hda.
  

• You can create floppy images using DD, althouth they are almost extinct:

  dd if=<image file> of=/dev/fd0

  . For example:

  dd if=/dev/fd0 of=floppy.img bs=18k 

  where /dev/fd0 should be the device for your raw floppy drive (_not_ /dev/floppy) and floppy.img the file you want to save the info to. You can then copy that file to somewhere you can read it with DOS, or maybe even zip it so it will fit onto a floppy ;). You should see something like the following to indicate that the image transfer was successful:

  2880+0 records in
  2880+0 records out

  If you see a smaller block count, your image did not transfer correctly. If this is the case, it will usually be accompanied by a disk error. After you make a disk, make sure to label it according to its contents.

• Getting around file size limitations using split

  When making images, it's quite easy to run up against various file size limitations. One way to work around a given file size limitation is to use the split command.

  # dd if=/dev/hda1 | gzip -c | split -b 2000m - /mnt/hdc1/backup.img.gz

  1. This example is using dd to take an image of the first partition on the first harddrive.
  2. The results are passed through to gzip for compression
     • The -c option switch is used to output the result to stdout.
  3. The compressed image is then piped to the split tool
     • The -b 2000m switch tells split how big to make the individual files. You can use k and m to tell switch kilobytes and megabytes (this option uses bytes by default).
     • The - option tells split to read from stdin. Otherwise, split would interpret the /mnt/hdc1... as the file to be split.
     • The /mnt/hdc1... is the prefix for the created files. Split will create files named backup.img.gz.aa, backup.img.gz.ab, etc.

  To restore the multi-file backup, do the following:

  # cat /mnt/hdc1/backup.img.gz.* | gzip -dc | dd of=/dev/hda1
  

  1. Cat recombines contents of the compressed and split image files to stdout, in order.
  2. Results are piped through gzip for decompression.
  3. And are then written to the first partition of the hard drive with dd.
• Creating empty disk images

  To create an empty disk image, to be used as the disk for an emulator for example, one can get data from /dev/zero. To create a 10mb image:

  $ dd if=/dev/zero of=myimage bs=1024 count=10240

  A clever alternative is:

  $ dd of=myimage bs=1024 count=0 seek=10240

  Here we don't write anything, not even zeroes, we just seek 10mb into the file and close it. The result is a sparse file that is implicitly full of 10mb of zeroes, but that takes no disk space. ls -l will report 10mb, while du and df will report 0. When the file is written to, either as an emulator disk or a loopback device, Linux will allocate disk space for the data. ls will still show 10mb, while du will gradually approach 10mb.

  For swap images, where it's more important to reserve the data than to save disk space, a non-sparse file is better.

• Copying from one tape drive to another. The following example copies from tape drive 0 to tape drive 1, using a common historical device naming convention.\

  dd if=/dev/rmt/0h of=/dev/rmt/1h

• Stripping the first 10 bytes from standard input. The following example strips the first 10 bytes from standard input:

  dd ibs=10 skip=1

• Reading a tape into an ASCII file. This example reads an EBCDIC tape blocked ten 80-byte EBCDIC card images per block into the ASCII file x:

  dd if=/dev/tape of=x ibs=800 cbs=80 conv=ascii,lcase

• Using conv=sync to write to tape. The following example uses conv=sync when writing to a tape:

  tar cvf - . | compress | dd obs=1024k of=/dev/rmt/0 conv=sync

• A typical task for dd is copying a floppy disk. As the common geometry of a 3.5" floppy is 18 sectors per track, two heads and 80 cylinders, an optimized dd command to read a floppy is:

  dd bs=2x80x18b if=/dev/fd0 of=/tmp/floppy.image 
  	1+0 records in
  	1+0 records out 

  The 18b specifies 18 sectors of 512 bytes, the 2x multiplies the sector size by the number of heads, and the 80x is for the cylinders--a total of 1474560 bytes. This issues a single 1474560-byte read request to /dev/fd0 and a single 1474560 write request to /tmp/floppy.image, whereas a corresponding cp command

  cp /dev/fd0 /tmp/floppy.image

  issues 360 reads and writes of 4096 bytes. While this may seem insignificant on a 1.44MB file, when larger amounts of data are involved, reducing the number of system calls and improving performance can be significant.

  This example also shows the factor capability in the GNU dd number specification. This has been around since before the Programmers Work Bench and, while not documented in the GNU dd man page, is present in the source and works just fine, thank you.

  To finish copying a floppy, the original needs to be ejected, a new diskette inserted, and another dd command issued to write to the diskette:

  Example 1-b : Copying to a 3.5" floppy
  	dd bs=2x80x18b < /tmp/floppy.image > /dev/fd0 
  	1+0 records in 
  	1+0 records out 

  Here is shown the stdin/stdout usage, in which respect dd is like most other utilities.
  

• The original need for dd came with the 1/2" tapes used to exchange data with other systems and boot and install Unix on the PDP/11. Those days are gone, but the 9-track format lives. To access the venerable 9-track, 1/2" tape, dd is superior. With modern SCSI tape devices, blocking and unblocking are no longer a necessity, as the hardware reads and writes 512-byte data blocks.

  However, the 9-track 1/2" tape format allows for variable length blocking and can be impossible to read with the cp command. The dd command allows for the exact specification of input and output block sizes, and can even read variable length block sizes, by specifying an input buffer size larger than any of the blocks on the tape. Short blocks are read, and dd happily copies those to the output file without complaint, simply reporting on the number of complete and short blocks encountered.

  Then there are the EBCDIC datasets transferred from such systems as MVS, which are almost always 80-character blank-padded Hollerith Card Images! No problem for dd, which will convert these to newline-terminated variable record length ASCII. Making the format is just as easy and dd again is the right tool for the job.

  dd bs=10240 cbs=80 conv=ascii,unblock if=/dev/st0 of=ascii.out
  40+0 records in
  38+1 records out 

  The fixed record length is specified by the cbs=80 parameter, and the input and output block sizes are set with bs=10240. The EBCDIC-to-ASCII conversion and fixed-to-variable record length conversion are enabled with the conv=ascii,noblock parameter.

  Notice the output record count is smaller than the input record count. This is due to the padding spaces eliminated from the output file and replaced with newline characters.
  

• Sometimes data arrives from sources in unusual formats. For example, every time I read a tape made on an SGI machine, the bytes are swapped. The dd command takes this in stride, swapping the bytes as required. The ability to use dd in a pipe with rsh means that the tape device on any *nix system is accessible, given the proper rlogin setup.

  rsh sgi.with.tape dd bs=256b if=/dev/rmt0 conv=swab | tar xvf - 

  The dd runs on the SGI and swaps the bytes before writing to the tar command running on the local host.
  

• Murphy's Law was postulated long before digital computers, but it seems it was specifically targeted for them. When you need to read a floppy or tape, it is the only copy in the universe and you have a deadline past due, that is when you will have a bad spot on the magnetic media, and your data will be unreadable. To the rescue comes dd, which can read all the good data around the bad spot and continue after the error is encountered. Sometimes this is all that is needed to recover the important data.

  dd bs=265b conv=noerror if=/dev/st0 of=/tmp/bad.tape.image

• The Linux kernel Makefiles use dd to build the boot image. In the Alpha Makefile /usr/src/linux/arch/alpha/boot/Makefile, the srmboot target issues the command:

  dd if=bootimage of=$(BOOTDEV) bs=512 seek=1 skip=1 

  This skips the first 512 bytes of the input bootimage file (skip=1) and writes starting at the second sector of the $(BOOTDEV) device (seek=1). A typical use of dd is to skip executable headers and begin writing in the middle of a device, skipping volume and partition data. As this can cause your disk to lose file system data, please test and use these applications with care.
  

• Really nerdy stuff:
  • view filesystems dd if=/proc/filesystems | hexdump -C | less
  • view all loaded modules dd if=/proc/kallsyms | hexdump -C | less
  • view interrupt table dd if=/proc/interrupts | hexdump -C | less
  • view system uptime (in seconds) dd if=/proc/uptime | hexdump -C | less
  • view partitions and sizes in kb dd if=/proc/partitions | hexdump -C | less
  • view mem stats dd if=/proc/meminfo | hexdump -C | less
• Post Learn the DD command by AwesomeMachine (Jan 2005) contains wealth of interesting examples:

  Linux DD

  The basic command structure is as follows:

  dd if=<source> of=<target> bs=<byte size> 

  bs "USUALLY" is some power of 2, and usually not less than 512 bytes (ie, 512, 1024, 2048, 4096, 8192, 16384, but can be any reasonable whole integer value. skip= seek= conv=<conversion>

  Source is the data being read. Target is where the data gets written.
  
  Warning!! If you reverse the source and target, you can wipe out a lot of data. This feature has inspired the nickname "dd" Data Destroyer. Warning!! Caution should be observed when using dd to duplicate encrypted partitions.
  
  Examples:

  duplicate one hard disk partition to another hard disk partition: Sda2 and sdb2 are partitions. You want to duplicate sda2 to sdb2.

  dd if=/dev/sda2 of=/dev/sdb2 bs=4096 conv=notrunc,noerror

  If sdb2 doesn't exist, dd will start at the beginning of the disk, and create it. Be careful with order of if and of. You can write a blank disk to a good disk if you get confused. If you duplicate a smaller partition to a larger one, using dd, the larger one will now be formatted the same as the smaller one. And there will be no space left on the drive. The way around this is to use

  rsync

  , as described below.
  
  To make an iso image of a CD: This duplicates sector for sector. MyCD.iso will be a hard disk image file of the CD.

  dd if=/dev/hdc of=/home/sam/myCD.iso bs=2048 conv=sync,notrunc

  You can mount the image like this:

  mkdir /mnt/myCD
  mount -o loop /home/sam/myCD.iso /mnt/myCD

  This will make the CD root directory the working directory, and display the CD root directory.

  cd /mnt/myCD

  This will duplicate a floppy disk to hard drive image file:

  dd if=/dev/fd0 of=/home/sam/floppy.image

  If you're concerned about spies taking the platters out of your hard drive, and scanning them using superconducting quantum-interference detectors, you can always add a "for" loop for US Government DoD approved secure hard disk erasure. Copy and paste the following two lines into a text editor.

  #!/bin/bash
  for n in `seq 7`; do dd if=/dev/urandom of=/dev/sda bs=8b conv=notrunc; done

  Save the file as anti_scqid.

  chmod +x anti_swqid

  Don't run the program until you want to wipe the drive.
  
  Best Laptop Backup: Purchase a laptop drive and an USB 2.0 drive enclosure (Total cost $100.00USD). Assemble the lappy drive into the external enclosure. Plug the external drive into the lappy USB port, and boot with The Knoppix live CD. Launch a terminal. This command will backup the existing drive:

  dd if=/dev/hda of=/dev/sda bs=64k conv=notrunc,noerror

  This command will restore from the USB drive to the existing drive:

  dd if=/dev/sda of=/dev/hda bs=64k conv=notrunc,noerror

  If the existing disk fails, you can boot from the external drive backup and have your system back instantaneously.
  
  This series will make a DVD backup of hard drive partition:

  dd if=/dev/hda3 of=/home/sam/backup_set_1.img bs=1M count=4430
  dd if=/dev/hda3 skip=4430 of=/home/sam/backup_set_2.img bs=1M count=4430 
  dd if=/dev/hda3 skip=8860 of=/home/sam/backup_set_3.img bs=1M count=4430

  And so on. This series will burn the images to DVD+/-R/RW:

  wodim -dev=/dev/hdc --driveropts=burnfree /home/sam/backup_set_1.img

  and so forth. To restore the from the backup, load the DVDs in order, and use commands like these:

   dd if=/media/dvd/backup_set_1.img of=/dev/hda3 bs=1M conv=sync,noerror

  Load another DVD

  dd if=/media/dvd/backup_set_2.img of=/dev/hda3 seek=4430 bs=1M conv=sync,noerror

  Load another DVD

  dd if=/media/dvd/backup_set_3.img of=/dev/hda3 seek=8860 bs=1M conv=sync,noerror

  and so forth.
  
  If you wrote chat messages and emails to another girl, on your girlfriend's computer, you can't be sure the files you deleted are unrecoverable. But you can make sure if anyone were to recover them, that you wouldn't get busted.

  dd if=/dev/sda | sed 's/Wendy/Janet/g' | dd of=/dev/sda

  Where every instance of Wendy is replaced by Janet, over every millimeter of disk. I picked names with the same number of characters, but you can pad a smaller name with blanks.
  
  This command will overwrite the drive with zeroes

  dd if=/dev/zero of=/dev/sda bs=4k conv=notrunc

  I just want to make sure my drive is really zeroed out!!

  dd if=/dev/sda | hexdump -C | grep [^00]

  ... will return output of every nonzero byte on the drive. Play around with it. Sometimes drives don't completely zero out on the first try.
  
  The following method of ouputting statistics applies to any dd command invocation. This is an example dd command so you can try it.

  /bin/dd if=/dev/zero of=/dev/null count=100MB

  When you want to know how far dd has gotten throwing 100MB of 512 byte blocks of zeroes into digital hell, open another terminal and do:

  ps aux | awk '/bin\/dd/ && !/awk/ {print $2}' | xargs kill -s USR1 $1

  In the terminal running the dd command you will find something like this:

  33706002+0 records in
  33706002+0 records out
  17257473024 bytes (17 GB) copied, 34.791 s, 496 MB/s

  If you enter the command again, you see more statistics:

  58596452+0 records in
  58596452+0 records out
  30001383424 bytes (30 GB) copied, 60.664 s, 495 MB/s

  Again

  74473760+0 records in
  74473760+0 records out
  38130565120 bytes (38 GB) copied, 77.3053 s, 493 MB/s

  and so on ... Until the command completes

  100000000+0 records in
  100000000+0 records out
  51200000000 bytes (51 GB) copied, 104.193 s, 491 MB/s

  How To Scan a dd Bitstream for Viruses and Malware:

   dd if=/home/sam/file.file | clamscan -

  Windows users will find help in the second post, way at the bottom
  
  FYI: duplicating smaller partition or drive to larger partition or drive; or vice versa:

  rsync -avH --exclude=/other_mount_point/ /mount_point/* /other_mount_point/

  You want to duplicate the root directory tree to another drive, but the other drive is larger. If you use dd, you will get a file system that is smaller then the larger destination drive. To duplicate files, not the file system: Format and mount the destination drive. Rsync will duplicate the files as files:

   rsync -avH --exclude=/mnt/destination_drive/  /* /mnt/destination_drive/

  You need to run:

  grub-install
  update-grub

  from a the rescue menu of an installation CD/DVD for the target to become bootable. If target was previously bootable, it remains bootable.
  
  Making a NTFS partition, is not easy without using Windows based tools. I was formatting an external drive for my brother, who uses MS Windows XP. I wasn't going to admit Linux couldn't make a NTFS partition.
  
  Make an ext3 partition on the drive. Open a hex editor and make a file containing

  07

  Save the file as file.bin. Change the ext3 partition to NTFS:

  dd if=/home/sam/file.bin of=/dev/sdb bs=1 seek=450 count=1

  Will change the partition type byte at offset

  0x1c2

  from Linux type:

  0x83

  , to NTFS type:

  0x07

  Please use a drive without important data on it. And, If you use a text editor to make the binary 07 file, you will ruin the existing partition table, because ascii 07 is two hexadecimal bytes (0x3037).
  
  The four primary partition type byte offsets are:

  0x1c2=450
  0x1d2=466
  0x1e2=482
  0x1f2=498

  If the dd seek= parameter is changed from 450 to a one of the other values, it will change partition (hd0,1), (hd0,2), or (hd0,3) to NTFS type, rather than partition (hd0,0).
  
  To be revised at a later date:
  To make a bootable flash drive: Download 50 MB Debian based distro here:
  http://sourceforge.net/projects/insert/
  
  Plug in the thumb drive into a USB port. Do:

  dmesg | tail

  Look where the new drive is, sdb1, or something similar. Do:

  dd if=/home/sam/insert.iso of=/dev/sdb ibs=4b obs=1b conv=notrunc,noerror

  Set the BIOS to USB boot, and boot.
  End to be revised
  
  This command will duplicate the MBR and boot sector of a floppy disk to hard drive image:

  dd if=/dev/fd0 of=/home/sam/MBRboot.image bs=512 count=2

  To clone an entire hard disk. /dev/sda is the source. /dev/sdb is the target:

  dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

  Do not reverse the intended source and target. It happens once in a while, especially to the inexperienced user. Notrunc means 'do not truncate the output file'. Noerror means to keep going if there is an error. Dd normally terminates on any I/O error.
  
  Duplicate MBR, but not partition table. This will duplicate the first 446 bytes of the hard drive to a file:

  dd if=/dev/sda of=/home/sam/MBR.image bs=446 count=1

  If you haven't already guessed, reversing the objects of if and of, on the dd command line, reverses the direction of the write.
  
  To wipe a hard drive: (Boot from a live CD distro to do this.)

  dd if=/dev/zero of=/dev/sda conv=notrunc

  This is useful for making the drive like new. Most drives have 0x00h written to every byte, from the factory.
  
  To overwrite all the free disk space on a partition (deleted files you don't want recovered):

  dd if=/dev/urandom of=/home/sam/bigfile.file

  When dd ouputs

  no room left on device

  all the free space has been overwritten with random characters. Delete the big file with

  rm bigfile.file

  Sometimes one wants to look inside a binary file, looking only for clues. The output of the command line:

  less /home/sam/file.bin

  is cryptic, because it's binary. For human readable output:

  dd if=/home/sam/file.bin | hexdump -C | less

  You may also use:

   dd if=/home/sam/file.file | strings -n 8 -t d | less

  Recover deleted JPEG files. Look at the header bytes of any JPEG.

  dd if=/home/sam/JPEG.jpg bs=1w count=2 | hexdump -C

  The last two bytes are the footer.

  dd if=JPEG.jpg | hexdump -C

  Using the JPEG header and footer bytes, search the drive. Command returns the offsets of the beginning and end of each deleted JPEG.

  dd if=/dev/sda3 | hexdump -C | "grep 'ff d8 ff e0' | 'ff d9'"

  If

  grep

  returned JPEG header bytes at offset:

  0xba0002f

  and footer bytes at offset:

  0xbaff02a

  Convert the hex offsets to decimal offsets, using one of the many logic capable calculators for Linux. Decimal offsets corresponding to the beginning and end of the JPEG are 195 035 183 and 196 079 658. (196 079 658) - (195 035 183) = rough idea of proper bs= and count= parameters. To find the proper count= figure: (<decinal offset of footer bytes> - <decimal offset of header bytes>) / <block size> = <number of blocks in the deleted JPEG file>. (195 035 183 – 196 079 658) = (1 044 475) / (bs=4096) = (254.998). That's really close to 255. If we could land exactly at the header bytes using bs=4096, we could use count=255. But I'm going to use count=257, because random chance dictates the probability of landing dead on the header bytes, using 2^x block size is remote. So we start reading before the header bytes.
  
  We need to use skip= parameter to skip to our start point: 195 035 183 / bs=4096 = 47 616.011. We always round down, so dd will start reading before the beginning of the file. In this case we round down to skip=47615. The following writes a file containing the JPEG with some unwanted bytes before and after.

  dd if=/dev/sda3 skip=47615 of=/home/sam/work_file.bin count=257 bs=4096

  This sequence yields the desired JPEG.

  hexdump -C work_file.bin | "grep 'ff d8 ff e0' | 'ff d9'"
  dd if=work_file.bin skip=<offset_of_first_header_byte_in_decimal_format> count=<offset_of_last_footer_byte_in_decimal_format +1> - <offset_of_first_header_byte_in_decimal_format> bs=1c of=JPG.jpg

  That's the way to get your hands dirty deep in digital data. But this process it automated in the file carving program, foremost.
  
  The principle of file carving negates the need for Linux undelete programs. So if your from a MS Windows world, don't google for linux undelete, but rather, foremost NEXT ...
  
  I put two identical drives in every one of my machines. Before I do anything that most probably spells disaster, like an untested command line in a root shell, that contains

  find / -regex ?*.???* -type f | xargs rm -f "$1"

  , I do:

  dcfldd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

  and duplicate my present working /dev/sda drive system to the /dev/sdb drive. If I wreck the installation on sda, I boot from a live CD distro, and do:

  dd if=/dev/sdb of=/dev/sda bs=4096 conv=notrunc,noerror

  And I get everything back exactly the same it was before whatever daring maneuver I was trying didn't work. You can really, really learn Linux this way, because you can't wreck what you have an exact duplicate of. You also might consider making the root partition separate from /home, and make /home big enough to hold the root partition, plus more. Then, To make a backup of root:

  dd if=/dev/sda2 (root) of=/home/sam/root.img bs=4096 conv=notrunc,noerror

  To write the image of root back to the root partition, if you messed up and can't launch the X server, or edited /etc/fstab, and can't figure out what you did wrong. It only takes a few minutes to restore a 15 GB root partition from an image file:

  dd if /home/sam/root.img of=/dev/sda2 (root) bs=4096 conv=notrunc,noerror

  
  
  How to make a swap file, or another swapfile on a running system:

  dd if=/dev/zero of=/swapspace bs=4k count=250000
  mkswap /swapspace
  swapon /swapspace

  This can solve out of memory issues due to memory leaks on servers that cannot easily be rebooted.
  
  How to pick proper block size:
  

  dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file
  dd if=/dev/zero bs=2048 count=500000 of=/home/sam/1Gb.file
  dd if=/dev/zero bs=4096 count=250000 of=/home/sam/1Gb.file
  dd if=/dev/zero bs=8192 count=125000 of=/home/sam/1Gb.file

  This method can also be used as a drive benchmark, to find strengths and weaknesses in hard drives:
  Read:

  dd if=/home/sam/1Gb.file bs=64k | dd of=/dev/null

  Write:

  dd if=/dev/zero bs=1024 count=1000000 of=/home/sam/1Gb.file

  When dd finishes it outputs (total size)/(total time). You get the idea.
  Play with 'bs=' and 'count=', always having them multiply out to the same toal size. You can calculate bytes/second like this: 1Gb/total seconds = Gb/s. You can get more realistic results using a 3Gb file.
  
  Rejuvenate a hard drive
  To cure input/output errors experienced when using dd. Over time the data on a drive, especially a drive that hasn't been used for a year or two, grows into larger magnetic flux points than were originally recorded. It becomes more difficult for the drive heads to decipher these magnetic flux points. This results in I/O errors. Sometimes sector 1 goes bad, resulting in a useless drive. Try:

  dd if=/dev/sda of=/dev/sda

  to rejuvenate the drive. Rewrites all the data on the drive in nice tight magnetic patterns that can then be read properly. The procedure is safe and economical.

  Make a file of 100 random bytes:

  dd if=/dev/urandom of=/home/sam/myrandom bs=100 count=1

  /dev/random produces only as many random bits as the entropy pool contains. This yields quality randomness for cryptographic keys. If more random bytes are required, the process stops until the entropy pool is refilled (waggling your mouse helps). /dev/urandom does not have this restriction. If the user demands more bits than are currently in the entropy pool, it produces them using a pseudo random number generator. Here, /dev/urandom is the Linux random byte device. Myrandom is a file.
  
  Randomize data over a file before deleting it:

  ls -l

  to find filesize.
  In this case it is 3769

  ls -l afile -rw------- ... 3769 Nov 2 13:41 <filename>

  dd if=/dev/urandom of=afile bs=3769 count=1 conv=notrunc

  duplicate a disk partition to a file on a different partition.
  
  Warning!! Do not write a partition image file to the same partition.

  dd if=/dev/sdb2 of=/home/sam/partition.image bs=4096 conv=notrunc,noerror

  This will make a file that is an exact duplicate of the sdb2 partition. You can substitue hdb, sda, hda, etc ... OR

  dd if=/dev/sdb2 ibs=4096 | gzip > partition.image.gz conv=noerror

  Makes a gzipped archive of the entire partition. To restore use:

   dd if=partition.image.gz | gunzip | dd of=/dev/sdb2

  For bzip2 (slower,smaller), substitute bzip2 and bunzip2, and name the file

  < filename >.bz2

  .Restore a disk partition from an image file.

  dd if=/home/sam/partition.image of=/dev/sdb2 bs=4096 conv=notrunc,noerror

  Convert a file to uppercase:

  dd if=filename of=filename conv=ucase

  Make a ramdrive:
  The Linux kernel makes a number a ramdisks you can make into ramdrives. You have to populate the drive with zeroes like so:

  dd if=/dev/zero of=/dev/ram7 bs=1k count=16384

  Populates a 16 MB ramdisk.

  mke2fs -m0 /dev/ram7 4096

  puts a file system on the ramdisk, turning it into a ramdrive. Watch this puppy smoke.

  debian:/home/sam # hdparm -t /dev/ram7
  /dev/ram7:
  Timing buffered disk reads: 16 MB in  0.02 seconds = 913.92 MB/sec

  You only need to do the timing once, because it's cool. Make the drive again, because hdparm is a little hard on ramdrives. You can mount the ramdrive with:

  mkdir /mnt/mem
  mount /dev/ram7 /mnt/mem

  Now you can use the drive like a hard drive. This is particularly superb for working on large documents or programming. You can duplicate the large file or programming project to the ramdrive, which on my machine is at least 27 times as fast as /dev/sda, and every time you save the huge document, or need to do a compile, it's like your machine is running on nitromethane. The only drawback is data security. The ramdrive is volatile. If you lose power, or lock up, the data on the ramdrive is lost. Use a reliable machine during clear skies if you use a ramdrive.
  
  Duplicate ram memory to a file:

  dd if=/dev/mem of=/home/sam/mem.bin bs=1024

  The device

  /dev/mem

  is your system memory. You can actually duplicate any block or character device to a file using dd. Memory capture on a fast system, with bs=1024 takes about 60 seconds, a 120 GB HDD about an hour, a CD to hard drive about 10 minutes, a floppy to a hard drive about 2 minutes. With dd, your floppy drive images will not change. If you have a bootable DOS diskette, and you save it to your HDD as an image file, when you restore that image to another floppy it will be bootable.
  
  Dd will print to the terminal window if you omit the

  of=/dev/output

  part.

  dd if=/home/sam/myfile

  will print the file myfile to the terminal window.
  
  To search the system memory:

  dd if=/dev/mem | strings | grep 'some-string-of-words-in-the-file-you-forgot-to-save-before-the-power-failed'

  If you need to cover your tracks quickly, put the following commands in a script to overwrite system ram with zeroes. Don't try this for fun.

  mkdir /mnt/mem
  mount -t ramfs /dev/mem /mnt/mem
  dd if=/dev/zero > /mnt/mem/bigfile.file

  This will overwrite all unprotected memory structures with zeroes, and freeze the machine so you have to reboot (Caution, this also prevents committment of the file system journal, and could trash the file system).
  
  You can get arrested in 17 states for doing this next thing. Make an AES encrypted loop device:

  dd if=/dev/urandom of=/home/sam/aes-drv bs=16065b count=100
  modprobe loop
  modprobe cryptoloop
  modprobe aes
  losetup -e aes /dev/loop1 ./aes-drv
  password:
  mkreiserfs /dev/loop1
  mkdir /aes
  mount -o loop,encryption=aes,acl ./aes-drv /aes
  password:
  mv /home/sam/porno /aes

  to get the porno on the aes drive image.

  umount /aes
  losetup -d /dev/loop1
  rmmod aes
  rmmod cryptoloop
  rmmod loop

  to make 'aes-drv' look like a 400 MB file of random bytes. Every time the lo interface is configured using losetup, according to the above, and the file 'aes-drv' is mounted, as above, the porno stash will be accessible in /aes/porno. You don't need to repeat the dd command, OR, the format with reiserfs, OR, the mv command. You only do those steps once. If you forget the password, there is no way to recover it besides guessing. Once the password is set, it can't be changed. To change the password, make a new file with the desired password, and move everything from the old file to the new file. Acl is a good mount option, because it allows use of acls. Otherwise your stuck with u,g,o and rwx.
  
  If you are curious about what might be on you disk drive, or what an MBR looks like, or maybe what is at the very end of your disk:

  dd if=/dev/sda count=1 | hexdump -C

  Will show you sector 1, or the MBR. The bootstrap code and partition table are in the MBR.
  To see the end of the disk you have to know the total number of sectors, and the MAS must be set equal to the MNA. The helix CD has a utility to set this correctly. In the dd command, your skip value will be one less than MNA of the disk. For a 120 GB Seagate SATA drives

  dd if=/dev/sda of=home/sam/myfile skip=234441646 bs=512

  ,
  So this reads sector for sector, and writes the last sector to myfile. Even with LBA addressing, disks still secretly are read in sectors, cylinders, and heads.
  There are 63 sectors per track, and 255 heads per cylinder. There is a total cylinder count. 512_bytes/sector*63_sectors/track*255heads=16065*512bytes/cylinder=8,225,280_bytes/cylinder. 63_sectors/track*255_heads=sectors/cylinder. With 234441647 total sectors, and 16065 sectors per cylinder, you get some trailing sectors which do not make up an entire cylinder: 14593.317584812_cylinders/drive. This leaves 5102 sectors which cannot be partitioned, because to be in a partition you have to be a whole cylinder. It's like having part of a person. That doesn't really count as a person. These become surplus sectors after the last partition. You can't ordinarily read past the last partition. But dd can. It's a good idea to check for anything writing to surplus sectors. For our Seagate 120 GB drive, 234,441,647_sectors/drive - 5102_surplus_sectors = 234,436,545 partitionable sectors.

  dd if=/dev/sda of=/home/sam/myfile skip=234436545

  writes the last 5102 sectors to myfile. Launch midnight commander (mc) to view the file. If there is something in there, you do not need it for anything. In this case you would write over it with random characters:

  dd if=/dev/urandom of=/dev/sda bs=512 seek=234436545

  Will overwrite the 5102 surplus sectors on our 120 GB Seagate drive.
  
  Block size:
  One cylinder in LBA mode = 255_heads*63_sectors/track=16065_sectors=16065*512_bytes=8,225,280_bytes. The b means '* 512'. 32130b represents a two cylinder block size. Cylinder block size always works to cover every sector in a partition, because partitions are made of a whole number of cylinders. One cylinder is 8,225,280 bytes. If you want to check out some random area of the disk:

  dd if=/dev/sda of=/home/sam/myfile bs=4096 skip=2000 count=1000

  Will give you 8,000 sectors in myfile, after the first 16,000 sectors. You can open that file with a hex editor, edit some of it, and write the edited part back to disk:

  dd if=/home/sam/myfile of=/dev/sda bs=4096 seek=2000 count=1000

  Image a partition to another machine:
  On source machine:

  dd if=/dev/hda bs=16065b | netcat < targethost-IP > 1234

  On target machine:

  netcat -l -p 1234 | dd of=/dev/hdc bs=16065b

  Variations on target machine:

  netcat -l -p 1234 | bzip2 > partition.img

  makes a compressed image file using bzip2 compression.

  netcat -l -p 1234 | gzip > partition.img

  makes a compressed image file using gzip compression. I back up a 100 GB lappy disk on a desktop drive, over a lan connection, and the 100 GB compresses to about 4.0 GB. Most of the drive is empty, so it's mostly zeroes. Repetitive zeroes compress well.
  Alert!! Don't hit enter yet. Hit enter on the target machine. THEN hit enter on the source machine.
  
  Netcat is a program, available by default, on most linux installations. It's a networking swiss army knife. In the preceding example, netcat and dd are piped to one another. One of the functions of the linux kernel is to make pipes. The pipe character looks like two little lines on top of one another, both vertical. Here is how this command behaves: This byte size is a cylinder. bs=16065b equals one cylinder on an LBA drive. The dd command is piped to netcat, which takes as its arguments the IP address of the target(like 192.168.0.1, or any IP address with an open port) and what port you want to use (1234).
  
  You can also use ssh.

  dd if=/dev/sdb2 | ssh sam@192.168.0.121 "sudo dd of=/home/sam/sdb2.img"

Do not use dd to copy files between file systems having different block sizes.

Using a blocked device to copy a file will result in extra nulls being added to the file to pad the final block to the block boundary.

When dd reads from a pipe, using the ibs=X and obs=Y operands, the output will always be blocked in chunks of size Y. When bs=Z is used, the output blocks will be whatever was available to be read from the pipe at the time.

When using dd to copy files to a tape device, the file size must be a multiple of the device sector size (for example, 512 Kbyte). To copy files of arbitrary size to a tape device, use tar(1) or cpio(1).

For SIGINT, dd writes status information to standard error before exiting. It takes the standard action for all other signals.

Old News ;-)

[Jan 05, 2012] Scalpel: A Frugal, High Performance File Carver

digitalforensicssolutions

Scalpel is a fast file carver that reads a database of header and footer definitions and extracts matching files or data fragments from a set of image files or raw device files. Scalpel is filesystem-independent and will carve files from FATx, NTFS, ext2/3, HFS+, or raw partitions. It is useful for both digital forensics investigation and file recovery.

Notes on Platforms

Linux

The preferred platform for using Scalpel is Linux.

Windows

Scalpel will also compile under Windows (32 or 64-bit) using mingw. If you'd like to try Scalpel on Windows without the bother of compiling it yourself, an executable and appropriate libraries are included in the distribution--just untar and go. Note that under Windows, the pthreads DLL must be present in the same directory as the Scalpel executable. Carving physical and logical devices directly under Windows (e.g., using \\.\physicaldrive0 as a target) is not supported in the current release.

Mac OS X

As of v1.53, Scalpel is supported on Mac OS X.

All platforms

As of v1.54, Scalpel supports carving files larger than 4GB on all platforms.

As of v1.60, Scalpel supports preview carving and other new carving modes. See the distribution for details.

As for v2.0, Scalpel supports regular expressions for headers and footers, minimum carve sizes, multithreading and asynchronous I/O, and beta-level support for GPU-accelerated file carving.

[Jan 19, 2011] Image software

LinkedIn

Leon Waldman

I always bet on dd + netcat to do imaging over network.

On the system where you will store the image (192.168.0.10):
netcat –l –p 7000 > file.iso

On the system to be cloned:
dd if=/dev/sda | netcat 192.168.0.10 7000 -q
where: dd if=volume_to_be_cloned| netcat ip_of_destination_ip port -q

You can even do it to full clone the system instead store it.

Give a look here:
http://digiassn.blogspot.com/2006/01/dd-over-netcat-for-cheap-ghost.html

[Mar 9, 2010] Copying Windows to a new drive, using linux - How-to-Guide by Ed Anderson

20 Mar 2005 | nilbus.com

Updated 18 May 2009 (NTFS Updates; device names)

This guide will show you how to copy an existing installation of Windows (or any other OS) from one drive to another - as long as the destination drive is the same size or larger.

This is a free and relatively easy method that will create a clone of your current hard disk, without having to buy any third party software.

1. Gathering tools
2. Physical Installation
3. Preparing new partition table
4. Copy the MBR
5. Copy the Partition
6. Resizing the Partition
7. FAQ

[Feb 11, 2009] Create a file with given size - Linux dd command

Feb 11, 2009 |unstableme.blogspot.com

"The command is:

dd if=/dev/zero of=testfile_10MB bs=10485760 count=1

	"1+0 records in
	1+0 records out

	10485760 bytes (10 MB) copied, 0.312 s, 33.6 MB/s"

[Oct 3, 2008] Red Hat Magazine This isn’t your grandpappy’s dd command

Looks like dd was slightly faster with 128K blocks. Python program might be reused for other purposes

Oct 02, 2008 | redhatmagazine.com

Block Size: 128 Throughput: 62.8 MB/s

Block Size: 256 Throughput: 61.8 MB/s

Block Size: 512 Throughput: 57.1 MB/s

Block Size: 1024 Throughput: 56.5 MB/s

We benchmarked the throughput of the disk by running the dd command with various block sizes from 128 KB to 1 MB. (Note: If you want to run the script on your own machine, make sure that the volume you use doesn’t contain any valuable data, because the data will be erased by the dd command. Remember, data loss makes grandpappy mad!)

For the benchmark, we wrote a Python script that uses the commands module to run and capture the output of the dd command. The script also uses the csv module to generate a comma-separated values file so that we can graph the results later. For this example, we chose to graph the results using the Google Chart API.

dd_chart on Flickr - Photo Sharing!

[Sep 12, 2008] Tips For Linux - How and when to use the dd command

[Oct 1, 2004 ] POWER TOOLS Performing Data Surgery

Data Dumping with dd

October 1, 2004 | Linux Magazine

dd  does low-level data transfer, byte-by-byte or block-by-block, with adjustable block sizes. It can also skip specified numbers of blocks in the input and/or output files, as well as converting data formats. All of those are handy for working with magnetic tape and disks. But it's also useful for many types of data transfers.

By default, dd reads the standard input and writes to the standard output. Input and output filenames, and other options too, are given in an unusual syntax without leading dash ( - ) characters.

For instance, to read a floppy disk and write its image to a file, you could type:

$ dd if=/dev/fd0 of=dosboot.img
2880+0 records in
2880+0 records out
$ ls -l dosboot.img
-rw-rw-r- ... 1474560 Nov 2 12:59 dosboot.img

The dd command line says, "Reading from the input file /dev/fd0, write all of the data to the file dosboot.img." dd doesn't try to find lines of data or individual files on the disk; it does a binary copy of the bytes from first to last. dd always tells you (on the standard error) how many times it read and wrote data. Above, it read 2,880 512-byte blocks. If you don't want to see this information -- or any error messages, either -- you can redirect dd's standard error to the Linux "bit bucket," /dev/null, by adding the Bourne shell operator 2>/dev/ null to the command line.

It's more efficient to specify a larger block size so the device drivers do a single read and write. There are lots of other options, and many of them start with conv= , like conv=unblock to replace trailing spaces in a block with a newline, and conv= swap to swap pairs of input bytes (which is needed with some tapes written on other types of hardware). But we'll leave that sort of optimization to you and the dd man page. Let's look at some less-obvious uses of this handy utility.

Stupid dd Tricks

Need a file with 100 arbitrary bytes -- for testing, for instance? The Linux device /dev/urandom (available since Linux 1.3.30) can supply as many pseudo-random bytes as you can read from it. To get just 100 bytes, set a block size of 1 byte with bs=1 and tell dd to stop after copying 100 "blocks" (here, that's 100 bytes):

$ dd if=/dev/urandom of=myrand bs=1 count=100

What's in that myrand file? The od utility can show you. (See the sidebar "What's In That File?")

If you need more-random data, try /dev/random instead. Reading data from /dev/random can take some time, though, as the random(4) man page explains. When you read from /dev /random, set a block size of 1.

Another use for dd is for "wiping" a text file before you delete it. Simply removing a Linux file (with rm, for instance) only deletes the inode that points to the data. A cracker with root access might read the raw disk (with dd!) and find the "deleted" file. We can use dd to write random data over the file before deleting it. Normally dd truncates a file before writing, so use conv=notrunc to make it write over the existing data. Set bs to the file size and count to 1 . For example:

% ls -l afile
-rw------- ... 3769 Nov  2 13:41 afile
% dd if=/dev/urandom of=afile \
  bs=3769 count=1 conv=notrunc
1+0 records in
1+0 records out
% rm afile

If you want to, you can repeat the "wiping" command several times with the C shell repeat command, the Z shell repeat loop, or simply use the history operator !!

[Oct 1, 2004 ] Moving your data to a backup device

faqs.org

The dd command can be used to put data on a disk, or get it off again, depending on the given input and output devices. An example:

gaby:~>dd if=images-without-dir.tar.gz of=/dev/fd0H1440
98+1 records in
98+1 records out

gaby~>dd if=/dev/fd0H1440 of=/var/tmp/images.tar.gz
2880+0 records in
2880+0 records out

gaby:~>ls /var/tmp/images*
/var/tmp/images.tar.gz

Note that the dumping is done on an unmounted device. Floppies created using this method will not be mountable in the file system, but it is of course the way to go for creating boot or rescue disks. For more information on the possibilities of dd, read the man pages.

This tool is part of the GNU fileutils package.

[Feb 13, 1998] SUMMARY How to build a Solaris bootable CDROM by James Hutchinson

sunmanagers.org

One would think this would be a simple task. Half the commerical packages proclaim that they can do it. But alas, for me, none of them worked.

The task was to create a system installer that would boot a sun, load up the appropriate system images and tools required for a server in our environment. Yes, this is a perfect job for jumpstart, but the systems would be distributed across the state and not connected via a fast enough pipe to use jumpstart effectivly. The other option was to build external disks with the proper images on them and boot from there. The cost of producing a few hundrad of these was prohibitive, thus the CDROM approach was taken.

Of the few responses I received from Sun Managers readers, all of them basicly said 'Its easy to clone the install disc!' or 'it just cant be done!'

Well. It is possible.

First, a breakdown of the process. When the command > boot cdrom < is givin to openboot the system actually looks at slice 2 to 5 for its bootblks depending on the machine type. A sun4c is slice 2 to an sun4u at slice 5. These bootblks redirect the system to slice 1 to load its kernel. The root partition is also stored in slice 1 and slice 0 is usr as well as packages. All slices save 0 are ufs, and 0 is hsfs.

This brings us to a small problem point. First we must boot from a UFS partition, as the bootblks all require that. Second, a CDROM has no label by default, thus its kinda hard to make partitions. And third, even if we do flush a UFS filesystem image off to the cdrom, the geometries will be all wrong, unless your staging disk just happens to have the same
geometries as the CDROM.

My solution is probally not the best. What would be required is an application that simply converts the UFS geometries from the staging disk uses to those used by the cdrom. I didnt really have time to create this so I did it as follows. Im sure I will get lots of flak for this
solution, but it does work.

1) Use dd to grab the first cylinder off the solaris boot cdrom. This contains a valid disk label and VTOC for the cdrom. Once this is created, our limitation is that we must work within the defines of this VTOC. You should be able to use prtvtoc on the cdrom to get a look at this VTOC, but this dosnt work if volmgr is running.

dd if=/dev/dsk/c0t6d0s0 of=cdrom.vtoc bs=512 count=1

2) Now use dd to grab the UFS slices from 1 to 5

for slice in 1 2 3 4 5
do
dd if=/dev/dsk/c0t6d0${slice} of=cdrom.s${slice}
done

3) Create a staging area and copy the parts of the usr filesystem (slice 0) off the cdrom into it. I started by copying cdrom:/export to it and then trimmed out the parts I didnt need like X and openwindows.

4) Add in all the things you need for your disc. For me, this ment a shell script that automated the build process, and images of all the data I wanted to move out. Make sure you donot go beyond te size of the slice 0 on the cdrom you started with. prtvtoc will should you this, if you cant get prtvtoc to work on the cdrom (sometimes it does, sometimes it dosnt) then use

> dd if=/dev/dsk/c0t6d0s0 of=/dev/null bs=512 <

Recordthe exact size of the partition in blocks as you will need it later.

5) Patch the slice 1 image (cdrom.s1) to start your custom application rather then the suninstall. This can be done by finding the break point you wish to use in the file cdrom:/sbin/sysconfig, selecting a unique set of chars in this file ( I used the string #***** S30sysid.net ) and then searching via a hexeditor or emacs in bin mode for that string.

Then find a comment line, change the first # and chars after to point to your script, then add a # line after. Example, I patched my image so the line

#***** S30sysid.net

became

exec build #ysid.net

I then put a script in my staging area into the /usr/bin dir. (the staging area will be made into slice 0 which is hsfs)

6) Using mkisofs or the tools that came with your cdrom burning package (HyCD worked well, as well as Gear) turn your staging area into a hsfs filesystem image, making sure that symbloic links are unmodified. HyCD required changing a default option that would have modified all the links.

7) Using dd, throw away the first block of this image.

dd if=image of=image.data bs=512 skip=1

8) Subtract the block count of slice 0 from the solaris cdrom from the image size above, add one to the answer. Say dd reported for the above step that your image was 500000 blocks, and your solaris cdrom has a size of 787840 for slice 0

787840 - ( 500000 + 1 ) = 287839

9) Feed this number into dd reading from /dev/zero to build a pad file.

dd if=/dev/zero of=image.pad bs=512 count=287839

10) Cat all of the image files together with the VTOC and the UFS slices.

cat cdrom.vtoc > image
cat image.data >> image
cat image.pad >> image
cat cdrom.s1 >> image
..
cat cdrom.s5 >> image

11) Burn this image to the cdrom drive using cdrecord, HyCD, Gear, etc

12) put it in a machine and test boot it and make sure it does what you need.

Thats about it. We have to go through the gyrations because UFS is geometry sensitive. We cannot take a image of a hard disk built UFS as all the cylinder groups will be off. The RIGHT way of doing this would be to build a tool that did the conversion for you and built an image up for burning. But that requires a bit more work, this gets the job done with a
minimal amount of strain. I am working on the above mentioned tool, but so far my progress has been to create lots of coasters. Sun has a tool called MakeDisc that does this job, or something similar, but I do not have a copy of it, so had to develop a method, while under a big gun, to do it in a very compressed amount of time. The only real limitation this
has is you can only store about 400 megs of information, of which around 70 or so are needed by usr in the hsfs partition. Plus you do not have to go through the pain of figuring out which parts of the system need to be moved to the memfs filesystem (cdrom is readonly, so dev, devices, etc need to be linked to /tmp)

If you wish to flame the procedure and tell me that I did it really stupid, please correct me! I would willing stop development of my tools to do same and do it the right way :) But my original question on how to do this went unanswered.

Have Fun
James

Recommended Links

• dd (Unix) - Wikipedia, the free encyclopedia
• Learn the DD command
• Marc's realm - Creating and using disk images mini-HOWTO
• test results for the disk imaging tool dd from the U.S. National Insitute of Justice
• link collection about forensics with opensource tools
• File system and disk images for testing digital forensic analysis and acquisition tools by Brian Carrier
• Using dd (instead of cdrecord and dvdrecord) to write bootable ISO images to DVD-RAM disks - CCP14 Linux Internet Server - CCP14 Web-Config Administration Information - CCP14 Homepage - Single Crystal and Powder Diffraction
• Dd - LQWiki
• Tips For Linux - How and when to use the dd command
• How to use floppy disks with Solaris
• BigAdmin Submitted Article- Backup and Recovery Utilities
• BigAdmin - Solaris (x86 Platform Edition) DCA Boot Diskette
• TAR DD or CPIO command
• Solaris Systems Administrator’s Guide, Second EditionAdministering Devices Try Copying Data from the Disk
• Burning a bootable CDRom for Solaris

Humor

It is jokingly said that dd stands for "destroy disk" or "delete data", since, being used for low-level operations on hard disks, a small mistake, such as reversing the if and of parameters, may accidentally render the entire disk unusable:

Tips For Linux - How and when to use the dd command

Also Murphy's Law was formulated long before digital computers, but it seems it was specifically targeted for them. When you need to read a floppy or tape, and it is the only copy in the universe the effect of a bad spot on the magnetic media is devastating. dd can read all the good data around the bad spot and continue after the error is encountered. Sometimes this is all that is needed to recover the important data.

Etc

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg  : C++ Humor : ARE YOU A BBS ADDICT? : Object oriented programmers of all nations :  C Humor : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor :  PseudoScience Related Humor : Networking Humor  : Shell Humor: Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : The Most Comprehensive Collection of Editor-related Humor : Microsoft plans to buy Catholic Church : Education Humor : IBM Humor :  Assembler-related Humor :  VIM Humor Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor : Best Russian Programmer Humor : Russian Musical Humor : The Perl Purity Test :  Politically Incorrect Humor : GPL-related Humor : OFM Humor : IDS Humor : Real Programmers Humor : Scripting Humor : Web Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor :

The Last but not Least

Copyright © 1996-2013 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Site uses AdSense so you need to be aware of Google privacy policy. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine. This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting hosting of this site with different providers to distribute and speed up access. Currently there are two functional mirrors: softpanorama.info (the fastest) and softpanorama.net.

Disclaimer:

The statements, views and opinions presented on this web page are those of the author and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

Last modified: January 31, 2013