|
Softpanorama
(slightly skeptical)
Open Source Software Educational Society |
May the
source be with you,
but remember the KISS principle ;-)
|
Unix cpio
The cpio command is one of standard Unix backup utilities. It stands
for "copy in/out." It is much less well known and rarely used Unix utility in comparison
with tar, but it is more powerful then tar due to its ability to handle links
and special files, append backup volumes and span tapes, allowing you to create
incremental backup sets and full systems backups without losing data integrity.
The input to copio is the list of files, so, for example, results of ls or find
commnad can be piped directly into cpio.
Cpio allows you to copy files into and out of a cpio archive.
If you use the -o option it archives pathname and file permissions information,
as well as the contents for each name of the file it gets as input.
The -o option copies out an archive. For example to back up the contents
of a directory you can use ls command. The device
file can be a tape drive, for example, /dev/mt0:
ls | cpio -o > /dev/mt0
You can also backup a partition to a file, for example
find /home -xdev -fstype ext3 | cpio -o >/backup/root080128.cpi
You can also backup key second level directories that should give you the opportunity
to restore the server:
for d in /boot /bin /sbin /dev
/lib64 /root /etc /home /opt /var ; do
echo $d
find $d -xdev
-fstype ext3 | cpio -o >/backup/$d\080128.cpi
sleep 10
done
In order to read from a cpio archive on a tape drive, you can do something
similar to the following example:
cpio -icdB < /dev/rmt0
where -i copies to an archive, c writes header information
in ASCII character form, d creates directories as needed, and B
blocks input to 512 bytes to the record.
You can also use the find command to see if a particular file is listed on your
tape (or disk) archive:
find . -cpio /dev/rmt/0m
Once again, consult your man pages for a more complete explanation of all the
options available.
If the archive was created using relative path names, the input files are built
as a directory within the current directory. If, however, the archive was created
with absolute path names, the same absolute paths are used to re-create the file.
Here is how to copy a directory's contents
The find command lets you copy the entire contents of a directory
while preserving the permissions, times, and ownership of every file and subdirectory.
To do so, combine find and the cpio command, like this:
Combining the find and cpio command
$ cd /path/to/source/dir
$ find . | cpio -pdumv /path/to/destination/dir
|
The cpio command is a command designed to copy files into and out of
a cpio or tar archive, automatically preserving permissions, times, and ownership
of files and subdirectories.All the files you specify are copied to the current
directory.
In this example, all files are copied from the tape in drive 0:
oak% cpio -icv < /dev/rmt/Ψ
Boot.chapter
Directory.chapter
Install.chapter
Intro.chapter
31 blocks
oak%
Retrieving a Subset of Files from a Tape
(cpio)
You can retrieve a subset of the files from the archive by specifying a pattern
to match using shell wildcard characters enclosed in quotation marks after the options:
- 1. Change to the directory where you want to put the files.
- 2. Insert the tape into the tape drive.
- 3. Type cpio -icv "* file
" < /dev/rmt/n and press Return. All the files that match
the pattern are copied to the current directory. You can specify multiple patterns,
but each must be enclosed in quotation marks.
In this example, all files that end in the suffix chapter are copied
from the tape in drive 0:
oak% cd /home/winsor/Book
oak% cpio -icv "*chapter" < /dev/rmt/Ψ
Boot.chapter
Directory.chapter
Install.chapter
Intro.chapter
31 blocks
oak%
See the cpio(1) manual page for more information.
07/11/2002 | onlamp.com
Let's start by creating a cpio
archive. In the last article, I created a test user account and
created a directory structure named www in this user's
home directory so I would have some files on which to practice using
the archiving utilities. I'll log in as the test user, cd
into the www directory, and see what happens if I use
the ls command with the cpio utility:cd www
ls | cpio -ov > backup.cpio
You'll note that I first cded into the directory
that contained the files I wished to archive. I used the ls
utility to make a list of the files in the current directory and
used a pipe (|) to send that list to the cpio utility.
The o switch invokes what is known as "copy out mode,"
which tells cpio to create an archive. The v
switch tells cpio to be verbose, meaning it will list
each file as it archives it. Finally, I used the >
redirector to write the results (the archive) to a file called
backup.cpio. I can call this file anything I like;
I chose to give it a cpio extension to remind me that it is a
cpio backup file. I can verify the file type using
the file utility:
file backup.cpio
backup.cpio: cpio archive
Instead of using the redirector, I could have also used the
F switch to specify which file to write the archive
to. So the following command will achieve the same results:
ls | cpio -ovF backup.cpio
Once the archive was created, cpio told me how many
blocks it wrote to the archive; in my case, it was 48 blocks.
So to create an archive, use the o switch or copy-out
mode. To either view or extract the contents of the archive, use
what is known as "copy-in mode." You invoke this mode by using the
i switch. If you just want to view the contents of
the archive, also include the t switch, which will
list the contents of the archive without extracting them:
cpio -it < backup.cpio
You'll note that this time I used the other redirector (<),
as I wanted the contents of the backup.cpio file to
be sent to the cpio utility. I can also include the
v switch, if I want to see a verbose listing of the
backup:
cpio -itv < backup.cpio
Remember that it is important to view the contents of an archive
before attempting to restore it, as you want to ensure that the
files don't begin with a /.
To restore this archive, I simply cd into the directory
to which I'd like to restore the archive, and repeat the above command
without the t switch. I'll cd back into
my home directory and create a directory named backupand
do the restore there:
cd
mkdir backup
cd backup
cpio -iv < ~/www/backup.cpio
You'll note something interesting if you try this exercise yourself;
if you use the ls -F command, you'll see that you did
indeed restore all of the files and directories that were in the
www directory. But if you cd into any
of those subdirectories, you'll note that they are empty. Even more
interestingly, if you try to remove any of those subdirectories,
you still have to use the R switch, as they are still
valid directories.
What happened here? Since the cpio utility received
its file list from the ls utility (and the ls
utility can only list the files in the current directory),
cpio was unaware of all of the files that existed below the
current directory. Remember, cpio will only archive
the files that are sent to it in a list. This may seem odd at first,
but it is an ideal way to archive just the files in the current
directory. In order to do this with the tar utility,
you would have to create an exclude file, as tar wants
to recursively copy everything in and below the current directory.
This doesn't mean that cpio can't archive recursively;
it simply means that if you want to just archive the current directory,
you use ls and if you want to archive recursively,
you use find instead.
Let's try that backup and restore again, this time using the
find utility. First, I'll remove the old backup and
empty out the backup directory:
rm www/backup.cpio
rm -R backup/*
Then I'll cd into the directory I wish to back up
(www) and archive its contents:
cd www
find -d . -print | cpio -ov > backup.cpio
When using the find utility with cpio,
it is always a good idea to include either the d or
the depth switch. Remember from the find
article that this switch prevented permissions from interfering
with a backup. When using this switch, either put -d
right after the word find and before the directory
to search (in this case, "."), or put the word
-depth after the directory to search, like so:
find . -depth -print | cpio -ov > backup.cpio
So as a recap on the find command, I told
find to search the current directory (".") and
to "print" its contents; the | was used to send those
contents to the cpio utility, which created an archive
(-o) and wrote that archive to a file called
backup.cpio. When I created this archive, I noted that
cpio wrote 43097 blocks, which is many more than the
48 I received with the ls command.
Now let's see what happens when I try to restore this archive:
cd ../backup
cpio -iv < ~/www/backup.cpio
I received an interesting message on my screen when I did this
restore:
<snip>
cpio: mod_tsunami/Makefile: No such file or directory
cpio: mod_tsunami/distinfo: No such file or directory
cpio: mod_tsunami/pkg-comment: No such file or directory
cpio: mod_tsunami/pkg-descr: No such file or directory
cpio: mod_tsunami/pkg-plist: No such file or directory
mod_tsunami
Makefile
.
43097 blocks
It looks like cpio read all 43097 blocks but complained
about missing files or directories. Indeed, if I do an ls
on any of the restored subdirectories, I'll discover that they are
once again empty! Don't worry, all of those files and directories
are in that archive file; I've simply demonstrated the default extraction
behaviour of cpio. Unlike tar, the
cpio utility does not recreate any directories during
the restore unless you specifically ask it to with the d
switch. And, unlike tar, the cpio utility
will not overwrite any existing files unless you specifically ask
it to with the u switch.
So let's try that restore again, this time using the d
switch to create the directories and the u switch to
overwrite the files I've already restored:
cpio -ivdu < ~/www/backup.cpio
This time I don't receive any error messages and I've successfully
restored all of the subdirectories and their files.
There're a few more switches you may consider using when backing
up and restoring with cpio. If I compare the modification
times of a file before it was archived and after it was restored,
I will see this:
ls -l www/zope/Makefile
-rw-r--r-- 1 test wheel 4308 May 11 09:53 www/zope/Makefile
ls -l backup/zope/Makefile
-rw-r--r-- 1 test wheel 4308 Jun 2 11:38 backup/zope/Makefile
ls -l www/backup.cpio
-rw-r--r-- 1 test wheel 22065664 Jun 2 10:39 www/backup.cpio
You'll note that the original file was created on May 11, that
it was backed up on June 2 at 10:39, and that it was restored on
June 2 at 11:38. If you want to preserve the file's original time,
include the a switch when creating the archive, and
the m switch when restoring the archive:
cd www
find -d . -print | cpio -ova > backup.cpio
cd ../backup
cpio -ivdm < ~/www/backup.cpio
If you try this and repeat the ls -l command, you'll
see that the original times of the archived files were kept intact.
The nice thing about using the find utility with
cpio is that you have all of find's switches
available to you, to fine-tune which files you would like to back
up. For example, if you'd like to do an incremental backup, use
find's -newer switch. In this example,
I'll back up all of the files in my home directory that have changed
since 11 PM on June 1st:
cd
touch -t 06012300 June1
find -d . -newer June1 -print | cpio -ova > backup.cpio
Here I used the touch utility to create an empty
file with a timestamp of month 06 day 01 time 2300, then I told
find to use the time on that file as the reference
point when searching the current directory. Alternatively, if I
wasn't concerned so much about the time as the date, I could have
used find's atime, ctime,
or mtime switches. And if I only want to archive files
of a certain size, I can use find's size
switch.
Before ending today's article, I'd also like to demonstrate
cpio's third mode, which is known as "copy-pass mode."
This is an interesting mode, as it archives and extracts in the
same command, making it ideal for copying one directory structure
and recreating it in another location.
Let's say I want to copy the www directory structure
from the home directory of the test user to the home directory of
the user genisis. I'll have to become the superuser, as I'll be
creating the archive in one user's home directory and recreating
it in another user's home directory:
su
Password:
cd ~test/www
find -d . -print | cpio -pvd ~genisis/www
Note that I first cded into the directory I wanted
to archive, in this case the www subdirectory of the
test user's home directory. Then, with the cpio command,
I invoked copy-pass mode with the p switch and specified
that I wanted the archive recreated in the www subdirectory
of the home directory of the user genisis.
If I run this command and then do an ls -l of genisis'
home directory, I'll see that I've successfully recreated the entire
www directory structure. However, I'll want to fine-tune
that above command as those restored files still belong to the user
"test." I'll repeat that command using the u switch
so it will overwrite that last restore, and I'll include the
R switch, which tells cpio to change the
ownership of the files as it recreates them:
find -d . -print | cpio -pvdu -R genisis ~genisis/www
When using the R switch, follow it by the name of
the user you wish to become the owner of the files, then follow
that by the name of the directory to restore the files to.
Finally, if I want to keep the original times of the files instead
of having them changed to the time the files were restored, I'd
also add the a and m switches:
find -d . -print | cpio -pvduam -R genisis ~genisis/www
This should get you started with the cpio command.
If you're planning on using cpio to copy between different
computers, you'll want to read its manpage first, as there may be
considerations, especially if the computers are running different
versions of Unix or different architectures.
In next week's article, I'll continue the archiver series by
introducing the pax command and, if space permits,
the dd command.
In this month's column, Eric moves beyond find to cover duplicating
files and directory trees using the versatile cpio command. cpio uses space
on tape more efficiently than tar and is an excellent alternative for creating
archives on platforms that do not have the GNU utilities available. Read on
for a thorough discussion of cpio and its three modes of operation: Pass-through,
Create and Extract.
Create an Archive
cpio creates archives differently than tar. Where tar
automatically recurses into subdirectories, cpio reads from stdin
a list of files & directories to archive; it does not automatically recurse
into directories.
To create an archive, give cpio the -o (copy out) command
line option. cpio will read a list of files & directories from
stdin, create the archive, & write the archive to stdout.
A good way to generate the list of files is the find program.
To archive everything in a directory, compress it with bzip2, &
write the results to a file, do this:
find dir -print |cpio -o |bzip2 >dir.cpio.bz2
That's the generic way to create an archive. On a Gnu/Linux system, you might
get a lot of ugly warnings about i-node numbers being truncated. The archive
will be fine, but it's never good to have unnecessary errors in the output;
the eye-sore might prevent you from seeing important error messages. To prevent
all those warnings, type this:
find dir -print |cpio -o -Hnewc |bzip2 >dir.cpio.bz2
A potential problem is that "-Hnewc" is not portable to all
implementations of cpio. So either you must know when it's okay to
use it or you must avoid using it & suffer with the gratuitous warning messages.
So far, we've created archives of all files in a directory tree. In other
words, we've reproduced the functionality of tar but at the cost of
more key strokes. Not very impressive. Since cpio reads a list of files
from stdin, we can do a lot more.
If you want to create a distribution archive of your source code, leaving
out object files (*.o), backup files (*~), and CVS
& RCS directories, just take advantage of the features of find that
you already know & love.
find dir \
-name "*.o" -o \
-name "*~" -o \
-name CVS -prune -o \
-name RCS -prune -o \
-print \
|cpio -o -Hnewc |bzip2 >dir.cpio.bz2
(I've broken the example into multiple lines for readability. You'd either
type the command on a single command line, or you'd break it into multiple lines,
as I've done, by including the back-slashes (\) literally.)
Need to backup just the files that have changed since your last backup yesterday?
Trivial!
find dir -ctime -1 -print \
|cpio -o -Hnewc |bzip2 >dir.cpio.bz2
By using find to generate the list of files, you can make cpio
archive any combination of files you want. It's easy to use find from
your own shell scripts, too, or you could even use your own programs to generate
the list of file names. cpio achieves great flexibility by leaving
the file-selection responsibilities to another program.
Advanced Features
Some (most? all?) cpio implementations are able to access file systems
& tapes through a cpio server on another host. A benefit there is that
you can use cpio to archive files from one host but write the archive
file to, say, the tape drive on another host. I've found this useful in cases
where I needed to backup large amounts of data to a tape drive, but the tape
drive was on a server that didn't have enough disk space to hold a temporary
copy of the entire archive, so I had to go directly to tape.
To use this feature, use the -O (that's a capital O)
command line option in conjunction with the user@host:pathname
method of specifying the destination file. See "man cpio" for details.
Similarly, you can use the -I command line option to extract
files from tape archives mounted on servers.
As cool as it sounds, this feature has some draw-backs. System-specific command
line options & device-file names are often necessary. For example, you might
have to force special block sizes with -B or --block-size,
or you might have to use system-specific device file names, such as /dev/st/n0a1bf00a
or something similarly incomprehensible. Also, systems sometimes behave as though
the communication between the client (your cpio process) & the server
are treated as text, so non-text characters & end-of-lines get mangled. In other
words, it sometimes just doesn't work.
In those cases, I've often made it work by using rsh and dd
explicitly. In other words:
find . -print |cpio -o -Hnewc \
|rsh server dd bs=32kb of=/dev/st0
(The values for block size (bs) & output file (of)
are system-specific, of course, & might differ for you.)
Using cpio to Back Up and Restore Files
You will find that this section on cpio uses much
of the same wording as in the previous section on tar. The two commands
have similar functionality, and only some specific details and examples change
the two sections. The tar and cpio sections are complete in
themselves, without complicated references to one another, but if you read them
both, you'll get a strong sense of dιjΰ vu.
Backing Up a Directory Subtree
cpio stands for copy in/out
and it enables you to back up files to tape, disk, or a disk file. Here is an
example of a basic cpio command to back up one complete directory:
$ cd /usr/fred/projects
$ find . -print | cpio -ocvaB > /dev/XXX?
First use cd to change the directory to back up.
Then use the find command to feed all the pathnames to cpio
to back them up.
The cpio command is one of the most commonly
used
Linux back up tools.
The cpio command has two unusual features
Unlike tar, in which the files to
back
up are typed in as part of the
command, cpio reads the files to work with from the standard input (in other
words, the screen).
This feature means that cpio must be used as part of a multiple command or with
a redirection pipe. Examples of this usage are shown in the tables below.
cpio must always be used with one of three flags. Flags are options that set
the mode in which the command runs. Only one flag can be used at a time, and
it must come before any other options. In addition, the choice of flags limits
the options that can be used. Each flag also has a gnu option that can used
in its place. The gnu option gives a convenient name for each flag: extract,
create, and pass- through.
If you want to know more options and how
to use check cpio
man page
Backing Up using the cpio Command
To do a backup, use cpio with a search command, such as find .
The basic structure is: find -name string -print | cpio -o options > directory
.
In this example:
The -name option for find lets you search for a string enclosed in double quotation
marks. Metacharacters can be used. The bar character ( | ) redirects the output
of find to cpio .
The -o flag sets cpio to create an archive file for backing up.
The target is a directory.
The > redirection operator redirects files to the location for the back up.
Typically, this location is on a removable device.
CPIO Command Examples
|
Examples |
What it does |
1) cd
/u/test
2) find . -print | cpio -ocv > /dev/fd0 |
Reads file names
using the find command and copies to the floppy drive (/dev/fd0). |
|
find
. -cpio /dev/fd0 -print |
Saves
files in current directory and writes this info to
floppy. Same command as above except much faster. |
1) cd
/u/test
2) cpio -icuvd < /dev/fd0 |
Restore
files and directories saved on the floppy device.
These files are restored under the current directory (/u/test) Only
if relative pathnames (./<filename>) were used. |
|
cpio
-itvcC1 < /dev/rmt0 |
List the table of
contents from a tape device. |
1) cd
/u/test
2) find . -print | cpio -dumpv /u/jerry |
Copies all files
FROM one directory TO another WITHOUT changing the permissions, owner/group
or modification date of the file. Use the following command to verify
that all files were copied: |
|
find
/u/test -print | wcfind /u/test1 -print | wc |
If the number of
files encountered is the same for both directories its safe to assume
that the directories are identical. NOTE: that the number of blocks
allocated to the SOURCE directory (/u/test) may be larger than the DESTINATION
directory (/u/test1), since compaction of the directory structure will
have occurred at the destination end. |
|
cpio
-imv /home/test/.profile < /dev/fd0 |
Selectively restore
the /home/test/.profile file from floppy |
|
cpio
-i "*.f" "*.c" </dev/fd0 |
Selectively restore
only the *.f and *.c files from floppy |
ONLamp.comIn
the previous article, I demonstrated the usage of the tar archiver
utility. This week I'll continue by introducing the cpio archiver
utility.
While both tar and cpio will achieve the same results,
the cpio utility approaches things a little bit differently. The
tar utility assumes that you want to recursively archive everything
under the specified directory or directories, meaning that you have to explicitly
tell tar if you want to exclude certain portions of that directory
structure. In contrast, the cpio utility expects to be explicitly
told which files or directories you wish to archive; this behavior is commonly
referred to as "receiving from standard input." In other words, cpio
expects to receive a list that contains one file per line, and if you remember
from the "Finding
Things in Unix" and "Find:
Part Two" articles, that is exactly the type of list that the find
utility creates. The ls utility can also create this type of list,
meaning that you will see either of the ls or the find
utilities used in conjunction with cpio. And since cpio
archives a list of files it receives from standard input, you usually use a
pipe (|) whenever you create an archive with the cpio utility.
The tar utility also assumes that you want to write the archive
to your first SCSI
tape drive, unless you explicitly specify a file using the f
switch. In contrast, the cpio utility writes to what is known as
standard output. This means that you will be using a redirector (either < or
>) whenever you are creating, listing, or extracting a cpio archive
file. Again, that file may be an actual file, or it may be your
floppy, or it may be a tape device, since in Unix everything is a file.
This may sound a bit more complicated at first, but a few examples should
convince you that it really isn't.
Let's start by creating a cpio archive. In the last article,
I created a test user account and created a directory structure named
www in this user's home directory so I would have some files on which
to practice using the archiving utilities. I'll log in as the test user,
cd into the www directory, and see what happens if
I use the ls command with the cpio utility:
cd www
ls | cpio -ov > backup.cpio
You'll note that I first cded into the directory that contained
the files I wished to archive. I used the ls utility to make a
list of the files in the current directory and used a pipe (|) to send that
list to the cpio utility. The o switch invokes what
is known as "copy out mode," which tells cpio to create an archive.
The v switch tells cpio to be verbose, meaning it
will list each file as it archives it. Finally, I used the > redirector
to write the results (the archive) to a file called backup.cpio.
I can call this file anything I like; I chose to give it a cpio extension to
remind me that it is a cpio
backup file. I can verify the file type using the file utility:
file backup.cpio
backup.cpio: cpio archive
Instead of using the redirector, I could have also used the F
switch to specify which file to write the archive to. So the following command
will achieve the same results:
ls | cpio -ovF backup.cpio
Once the archive was created, cpio told me how many blocks it
wrote to the archive; in my case, it was 48 blocks.
So to create an archive, use the o switch or copy-out mode.
To either view or extract the contents of the archive, use what is known as
"copy-in mode." You invoke this mode by using the i switch. If
you just want to view the contents of the archive, also include the t
switch, which will list the contents of the archive without extracting them:
cpio -it < backup.cpio
You'll note that this time I used the other redirector (<),
as I wanted the contents of the backup.cpio file to be sent to
the cpio utility. I can also include the v switch,
if I want to see a verbose listing of the backup:
cpio -itv < backup.cpio
Remember that it is important to view the contents of an archive before attempting
to restore it, as you want to ensure that the files don't begin with a
/.
To restore this archive, I simply cd into the directory to which
I'd like to restore the archive, and repeat the above command without the
t switch. I'll cd back into my home directory and
create a directory named backupand do the restore there:
cd
mkdir backup
cd backup
cpio -iv < ~/www/backup.cpio
You'll note something interesting if you try this exercise yourself; if you
use the ls -F command, you'll see that you did indeed restore all
of the files and directories that were in the www directory. But
if you cd into any of those subdirectories, you'll note that they
are empty. Even more interestingly, if you try to remove any of those subdirectories,
you still have to use the R switch, as they are still valid directories.
What happened here? Since the cpio utility received its file
list from the ls utility (and the ls utility can only
list the files in the current directory), cpio was unaware of all
of the files that existed below the current directory. Remember, cpio
will only archive the files that are sent to it in a list. This may seem odd
at first, but it is an ideal way to archive just the files in the current directory.
In order to do this with the tar utility, you would have to create
an exclude file, as tar wants to recursively copy everything in
and below the current directory.
This doesn't mean that cpio can't archive recursively; it simply
means that if you want to just archive the current directory, you use
ls and if you want to archive recursively, you use find
instead.
Let's try that
backup and restore again, this time using the find utility.
First, I'll remove the old backup and empty out the backup directory:
rm www/backup.cpio
rm -R backup/*
Then I'll cd into the directory I wish to back up (www)
and archive its contents:
cd www
find -d . -print | cpio -ov > backup.cpio
When using the find utility with cpio, it is always
a good idea to include either the d or the depth switch.
Remember from the find article that this switch prevented permissions
from interfering with a backup. When using this switch, either put -d
right after the word find and before the directory to search
(in this case, "."), or put the word -depth after
the directory to search, like so:
find . -depth -print | cpio -ov > backup.cpio
So as a recap on the find command, I told find
to search the current directory (".") and to "print" its contents;
the | was used to send those contents to the cpio
utility, which created an archive (-o) and wrote that archive to
a file called backup.cpio. When I created this archive, I noted
that cpio wrote 43097 blocks, which is many more than the 48 I
received with the ls command.
Now let's see what happens when I try to restore this archive:
cd ../backup
cpio -iv < ~/www/backup.cpio
I received an interesting message on my screen when I did this restore:
<snip>
cpio: mod_tsunami/Makefile: No such file or directory
cpio: mod_tsunami/distinfo: No such file or directory
cpio: mod_tsunami/pkg-comment: No such file or directory
cpio: mod_tsunami/pkg-descr: No such file or directory
cpio: mod_tsunami/pkg-plist: No such file or directory
mod_tsunami
Makefile
.
43097 blocks
It looks like cpio read all 43097 blocks but complained
about missing files or directories. Indeed, if I do an ls on
any of the restored subdirectories, I'll discover that they are once again
empty! Don't worry, all of those files and directories are in that archive
file; I've simply demonstrated the default extraction behaviour of
cpio. Unlike tar, the cpio utility does
not recreate any directories during the restore unless you specifically
ask it to with the d switch. And, unlike tar,
the cpio utility will not overwrite any existing files unless
you specifically ask it to with the u switch.
So let's try that restore again, this time using the d switch
to create the directories and the u switch to overwrite the
files I've already restored:
cpio -ivdu < ~/www/backup.cpio
This time I don't receive any error messages and I've successfully restored
all of the subdirectories and their files.
There're a few more switches you may consider using when
backing up and restoring with cpio. If I compare the modification
times of a file before it was archived and after it was restored, I will
see this:
ls -l www/zope/Makefile
-rw-r--r-- 1 test wheel 4308 May 11 09:53 www/zope/Makefile
ls -l backup/zope/Makefile
-rw-r--r-- 1 test wheel 4308 Jun 2 11:38 backup/zope/Makefile
ls -l www/backup.cpio
-rw-r--r-- 1 test wheel 22065664 Jun 2 10:39 www/backup.cpio
You'll note that the original file was created on May 11, that it was
backed up on June 2 at 10:39, and that it was restored on June 2 at 11:38.
If you want to preserve the file's original time, include the a
switch when creating the archive, and the m switch when restoring
the archive:
cd www
find -d . -print | cpio -ova > backup.cpio
cd ../backup
cpio -ivdm < ~/www/backup.cpio
If you try this and repeat the ls -l command, you'll see
that the original times of the archived files were kept intact.
The nice thing about using the find utility with cpio
is that you have all of find's switches available to you, to
fine-tune which files you would like to
back up. For example, if you'd like to do an incremental backup, use
find's -newer switch. In this example, I'll back
up all of the files in my home directory that have changed since 11 PM on
June 1st:
cd
touch -t 06012300 June1
find -d . -newer June1 -print | cpio -ova > backup.cpio
Here I used the touch utility to create an empty file with
a timestamp of month 06 day 01 time 2300, then I told find
to use the time on that file as the reference point when searching the current
directory. Alternatively, if I wasn't concerned so much about the time as
the date, I could have used find's atime,
ctime, or mtime switches. And if I only want to
archive files of a certain size, I can use find's size
switch.
Before ending today's article, I'd also like to demonstrate cpio's
third mode, which is known as "copy-pass mode." This is an interesting mode,
as it archives and extracts in the same command, making it ideal for copying
one directory structure and recreating it in another location.
Let's say I want to copy the www directory structure from
the home directory of the test user to the home directory of the user genisis.
I'll have to become the superuser, as I'll be creating the archive in one
user's home directory and recreating it in another user's home directory:
su
Password:
cd ~test/www
find -d . -print | cpio -pvd ~genisis/www
Note that I first cded into the directory I wanted to archive,
in this case the www subdirectory of the test user's home directory.
Then, with the cpio command, I invoked copy-pass mode with
the p switch and specified that I wanted the archive recreated
in the www subdirectory of the home directory of the user genisis.
If I run this command and then do an ls -l of genisis' home
directory, I'll see that I've successfully recreated the entire www
directory structure. However, I'll want to fine-tune that above command
as those restored files still belong to the user "test." I'll repeat that
command using the u switch so it will overwrite that last restore,
and I'll include the R switch, which tells cpio
to change the ownership of the files as it recreates them:
find -d . -print | cpio -pvdu -R genisis ~genisis/www
When using the R switch, follow it by the name of the user
you wish to become the owner of the files, then follow that by the name
of the directory to restore the files to.
Finally, if I want to keep the original times of the files instead of
having them changed to the time the files were restored, I'd also add the
a and m switches:
find -d . -print | cpio -pvduam -R genisis ~genisis/www
This should get you started with the cpio command. If you're
planning on using cpio to copy between different
computers, you'll want to read its manpage first, as there may be considerations,
especially if the computers are running different versions of Unix or different
architectures.
In next week's article, I'll continue the archiver series by introducing
the pax command and, if space permits, the dd
command.
In case of broken links
please try to use Google search. If you find the page please notify
us about new location
AIX Commands Reference
CPIO How-To
by Gene Michael Stover's
Is tar-cpio a good backup program
Reference
GNU cpio
--- The Detailed Node Listing ---
Invoking cpio
1 Introduction
GNU cpio copies files into or out of a cpio or tar archive, The archive can
be another file on the disk, a magnetic tape, or a pipe.
GNU cpio supports the following archive formats: binary, old ASCII, new ASCII,
crc, HPUX binary, HPUX old ASCII, old tar, and POSIX.1 tar. The tar format is
provided for compatibility with the tar program. By default, cpio creates binary
format archives, for compatibility with older cpio programs. When extracting
from archives, cpio automatically recognizes which kind of archive it is reading
and can read archives created on machines with a different byte-order.
2 Tutorial
GNU cpio performs three primary functions. Copying files to an archive, Extracting
files from an archive, and passing files to another directory tree. An archive
can be a file on disk, one or more floppy disks, or one or more tapes.
When creating an archive, cpio takes the list of files to be processed from
the standard input, and then sends the archive to the standard output, or to
the device defined by the -F option.
See Copy-out mode. Usually
find or ls is used to provide this list to the standard input. In the following
example you can see the possibilities for archiving the contents of a single
directory.
| % ls | cpio -ov > directory.cpio |
The -o option creates the archive,
and the -v option prints the names
of the files archived as they are added. Notice that the options can be put
together after a single - or can be
placed separately on the command line. The >
redirects the cpio output to the file directory.cpio.
If you wanted to archive an entire directory tree, the find command can provide
the file list to cpio:
| find . -print -depth | cpio -ov > tree.cpio |
This will take all the files in the current directory, the directories below
and place them in the archive tree.cpio. Again the
-o creates an archive, and the
-v option shows you the name of the files
as they are archived. See Copy-out
mode. Using the . in the find statement
will give you more flexibility when doing restores, as it will save file names
with a relative path vice a hard wired, absolute path. The
-depth option forces find
to print of the entries in a directory before printing the directory itself.
This limits the effects of restrictive directory permissions by printing the
directory entries in a directory before the directory name itself.
Extracting an archive requires a bit more thought because cpio will not create
directories by default. Another characteristic, is it will not overwrite existing
files unless you tell it to.
| % cpio -iv < directory.cpio |
This will retrieve the files archived in the file directory.cpio and place
them in the present directory. The -i
option extracts the archive and the -v
shows the file names as they are extracted. If you are dealing with an archived
directory tree, you need to use the -d
option to create directories as necessary, something like:
This will take the contents of the archive tree.cpio and extract it to the
current directory. If you try to extract the files on top of files of the same
name that already exist (and have the same or later modification time) cpio
will not extract the file unless told to do so by the -u option. See
Copy-in mode.
In copy-pass mode, cpio copies files from one directory tree to another,
combining the copy-out and copy-in steps without actually using an archive.
It reads the list of files to copy from the standard input; the directory into
which it will copy them is given as a non-option argument. See
Copy-pass mode.
| % find . -depth -print0 | cpio --null -pvd new-dir |
The example shows copying the files of the present directory, and sub-directories
to a new directory called new-dir. Some new options are the
-print0 available with GNU find, combined
with the --null option of cpio. These
two options act together to send file names between find and cpio, even if special
characters are embedded in the file names. Another is
-p, which tells cpio to pass the files it
finds to the directory new-dir.
3 Invoking cpio
3.1 Copy-out mode
In copy-out mode, cpio copies files into an archive. It reads a list of filenames,
one per line, on the standard input, and writes the archive onto the standard
output. A typical way to generate the list of filenames is with the find command;
you should give find the -depth option to minimize problems with permissions
on directories that are unreadable. See Options.
cpio {-o|--create} [-0acvABLV] [-C bytes] [-H format]
[-M message] [-O [[user@]host:]archive] [-F [[user@]host:]archive]
[--file=[[user@]host:]archive] [--format=format]
[--message=message][--null] [--reset-access-time] [--verbose]
[--dot] [--append] [--block-size=blocks] [--dereference]
[--io-size=bytes] [--rsh-command=command] [--help] [--version]
< name-list [> archive]
3.2 Copy-in mode
In copy-in mode, cpio copies files out of an archive or lists the archive
contents. It reads the archive from the standard input. Any non-option command
line arguments are shell globbing patterns; only files in the archive whose
names match one or more of those patterns are copied from the archive. Unlike
in the shell, an initial . in a filename
does match a wildcard at the start of a pattern, and a /
in a filename can match wildcards. If no patterns are given, all files are extracted.
See Options.
cpio {-i|--extract} [-bcdfmnrtsuvBSV] [-C bytes] [-E file]
[-H format] [-M message] [-R [user][:.][group]]
[-I [[user@]host:]archive] [-F [[user@]host:]archive]
[--file=[[user@]host:]archive] [--make-directories]
[--nonmatching] [--preserve-modification-time]
[--numeric-uid-gid] [--rename] [--list] [--swap-bytes] [--swap]
[--dot] [--unconditional] [--verbose] [--block-size=blocks]
[--swap-halfwords] [--io-size=bytes] [--pattern-file=file]
[--format=format] [--owner=[user][:.][group]]
[--no-preserve-owner] [--message=message] [--help] [--version]
[--no-absolute-filenames] [--sparse] [-only-verify-crc] [--to-stdout]
[-quiet] [--rsh-command=command] [pattern...] [< archive]
3.3 Copy-pass mode
In copy-pass mode, cpio copies files from one directory tree to another,
combining the copy-out and copy-in steps without actually using an archive.
It reads the list of files to copy from the standard input; the directory into
which it will copy them is given as a non-option argument. See
Options.
cpio {-p|--pass-through} [-0adlmuvLV] [-R [user][:.][group]]
[--null] [--reset-access-time] [--make-directories] [--link]
[--preserve-modification-time] [--unconditional] [--verbose]
[--dot] [--dereference] [--owner=[user][:.][group]] [--sparse]
[--no-preserve-owner] [--help] [--version] destination-directory
< name-list
3.4 Options
-0 --null - Read a list of filenames terminated by a null character, instead of
a newline, so that files whose names contain newlines can be archived. GNU
find is one way to produce a list of null-terminated filenames. This option
may be used in copy-out and copy-pass modes.
-a --reset-access-time - Reset the access times of files after reading them, so that it does
not look like they have just been read.
-A --append - Append to an existing archive. Only works in copy-out mode. The archive
must be a disk file specified with the -O
or -F (--file)
option.
-b --swap - Swap both halfwords of words and bytes of halfwords in the data. Equivalent
to -sS. This option may be used in copy-in mode. Use this option to convert
32-bit integers between big-endian and little-endian machines.
-B - Set the I/O block size to 5120 bytes. Initially the block size is 512
bytes.
--block-size=block-size - Set the I/O block size to block-size * 512 bytes.
-c - Use the old portable (ASCII) archive format.
-C io-size --io-size=io-size - Set the I/O block size to io-size bytes.
-d --make-directories - Create leading directories where needed.
-E file --pattern-file=file - Read additional patterns specifying filenames to extract or list from
file. The lines of file are treated as if they had
been non-option arguments to cpio. This option is used in copy-in mode,
-f --nonmatching - Only copy files that do not match any of the given patterns.
-F archive --file=archive - Archive filename to use instead of standard input or output. To use
a tape drive on another machine as the archive, use a filename that starts
with hostname:, where
hostname is the name or IP address of the machine. The hostname
can be preceded by a username and an @
to access the remote tape drive as that user, if you have permission to
do so (typically an entry in that user's ~/.rhosts
file).
--force-local - With -F,
-I, or -O,
take the archive file name to be a local file even if it contains a colon,
which would ordinarily indicate a remote host name.
-H format --format=format - Use archive format format. The valid formats are listed below;
the same names are also recognized in all-caps. The default in copy-in mode
is to automatically detect the archive format, and in copy-out mode is bin.
- bin
- The obsolete binary format.
- odc
- The old (POSIX.1) portable format.
- newc
- The new (SVR4) portable format, which supports file systems having
more than 65536 i-nodes.
- crc
- The new (SVR4) portable format with a checksum added.
- tar
- The old tar format.
- ustar
- The POSIX.1 tar format. Also recognizes GNU tar archives, which
are similar but not identical.
- hpbin
- The obsolete binary format used by HPUX's cpio (which stores device
files differently).
- hpodc
- The portable format used by HPUX's cpio (which stores device files
differently).
-i --extract - Run in copy-in mode. See
Copy-in mode.
-I archive - Archive filename to use instead of standard input. To use a tape drive
on another machine as the archive, use a filename that starts with hostname:,
where hostname is the name or IP address of the remote host.
The hostname can be preceded by a username and an @
to access the remote tape drive as that user, if you have permission to
do so (typically an entry in that user's ~/.rhosts
file).
-k - Ignored; for compatibility with other versions of cpio.
-l --link - Link files instead of copying them, when possible.
-L --dereference - Copy the file that a symbolic link points to, rather than the symbolic
link itself.
-m --preserve-modification-time - Retain previous file modification times when creating files.
-M message --message=message - Print message when the end of a volume of the backup media
(such as a tape or a floppy disk) is reached, to prompt the user to insert
a new volume. If message contains the string %d,
it is replaced by the current volume number (starting at 1).
-n --numeric-uid-gid - Show numeric UID and GID instead of translating them into names when
using the --verbose option.
--no-absolute-filenames - Create all files relative to the current directory in copy-in mode,
even if they have an absolute file name in the archive.
--no-preserve-owner - Do not change the ownership of the files; leave them owned by the user
extracting them. This is the default for non-root users, so that users on
System V don't inadvertantly give away files. This option can be used in
copy-in mode and copy-pass mode
-o --create - Run in copy-out mode. See
Copy-out mode.
-O archive - Archive filename to use instead of standard output. To use a tape drive
on another machine as the archive, use a filename that starts with hostname:,
where hostname is the name or IP address of the machine. The
hostname can be preceded by a username and an @
to access the remote tape drive as that user, if you have permission to
do so (typically an entry in that user's ~/.rhosts
file).
--only-verify-crc - Verify the CRC's of each file in the archive, when reading a CRC format
archive. Don't actually extract the files.
-p --pass-through - Run in copy-pass mode. See
Copy-pass mode.
--quiet - Do not print the number of blocks copied.
-r --rename - Interactively rename files.
-R owner --owner owner - In copy-in and copy-pass mode, set the ownership of all files created
to the specified owner (this operation is allowed only for the
super-user). In copy-out mode, store the supplied owner information in the
archive.
The argument can be either the user name or the user name and group name,
separated by a dot or a colon, or the group name, preceeded by a dot or
a colon, as shown in the examples below:
cpio --owner smith
cpio --owner smith:
cpio --owner smith:users
cpio --owner :users
If the group is omitted but the :
or . separator is given, as in
the second example. the given user's login group will be used.
--rsh-command=command - Notifies cpio that is should use command to communicate with
remote devices.
-s --swap-bytes - Swap the bytes of each halfword (pair of bytes) in the files. This option
can be used in copy-in mode.
-S --swap-halfwords - Swap the halfwords of each word (4 bytes) in the files. This option
may be used in copy-in mode.
--sparse - Write files with large blocks of zeros as sparse files. This option
is used in copy-in and copy-pass modes.
-t --list - Print a table of contents of the input.
--to-stdout - Extract files to standard output. This option may be used in copy-in
mode.
-u --unconditional - Replace all files, without asking whether to replace existing newer
files with older files.
-v --verbose - List the files processed, or with -t,
give an ls -l style table of contents
listing. In a verbose table of contents of a ustar archive, user and group
names in the archive that do not exist on the local system are replaced
by the names that correspond locally to the numeric UID and GID stored in
the archive.
-V --dot - Print a . for each file processed.
--version - Print the cpio program version number and exit.
4 Magnetic Media
Archives are usually written on removable mediatape cartridges, mag tapes,
or floppy disks.
The amount of data a tape or disk holds depends not only on its size, but
also on how it is formatted. A 2400 foot long reel of mag tape holds 40 megabytes
of data when formated at 1600 bits per inch. The physically smaller EXABYTE
tape cartridge holds 2.3 gigabytes.
Magnetic media are re-usableonce the archive on a tape is no longer needed,
the archive can be erased and the tape or disk used over. Media quality does
deteriorate with use, however. Most tapes or disks should be disgarded when
they begin to produce data errors.
Magnetic media are written and erased using magnetic fields, and should be
protected from such fields to avoid damage to stored data. Sticking a floppy
disk to a filing cabinet using a magnet is probably not a good idea.
5 Reporting bugs or suggestions
It is possible you will encounter a bug in cpio.
If this happens, we would like to hear about it. As the purpose of bug reporting
is to improve software, please be sure to include maximum information when reporting
a bug. The information needed is:
- Version of the package you are using.
- Compilation options used when configuring the package.
- Conditions under which the bug appears.
Send your report to <bug-cpio@gnu.org>. Allow us a couple of days to answer.
Concept Index
Copyright © 1996-2009 by Dr. Nikolai Bezroukov.
www.softpanorama.org was
created as a service to the UN Sustainable Development Networking Programme (SDNP)
in the author free time.
Submit
comments This document is an industrial compilation designed and created
exclusively for educational use and is placed under the copyright of the
Open Content License(OPL).
Site uses AdSense so you need to be aware of Google privacy policy. Original materials copyright belong to respective owners. Quotes are made
for educational purposes only in compliance with the fair use doctrine.
Disclaimer:
- The statements, views and opinions presented on
this web page are those of the author and are not endorsed by, nor do they necessarily
reflect, the opinions of the author present and former employers, SDNP or any other
organization the author may be associated with.
- We do not warrant the correctness of the information provided or its
fitness for any purpose
- In no way this site is associated with or endorse cybersquatters
using
the term "softpanorama" with other main or country domains (e.g. softpanorama.com) with
bad faith intent to profit from the goodwill belonging to
someone else.
Last modified:
August 10, 2009
