Softpanorama Bulletin. Vol 31, No. 01 (January, 2019))

Creative uses of rm

Unintended consequences, or wiping out much more then you intended to do

Version 1.8 (Aug 27, 2019)

Copyright 2006-2019, Nikolai Bezroukov

NOTE: This is copyrighted unpublished work based on original research by the author and presented as a preprint. All right reserved.

• Introduction
• Steps that you can do to reduce the chances of the major damage to the data
• Use a wrapper like "saferm"
• Classic blunders with RM command
• Classic blunders in scripts with rm command
• Backup files before executing rm command if you deleting a directory and it is not that large
• More on blunders with the deleting a directory from the backup
• Important class of subtle Unix errors: Dot-star errors
• Using "convenience" symlinks to other (typically deeply nested) directories inside a directories and forgetting about them
• Disastrous typos in name of the directory or regex -- unintended space
• Root deletion protection
• Using echo command to see expansion of your wildcards, if any
• Using file starting with - as stoppers for unintended use of rm. Undeletable files
• Blunders in using rm with find

Introduction

The most enduring principles in the universe are accident and error

SNAFU is an acronym that is widely used to stand
for the sarcastic expression Situation Normal: All Fucked Up.
It is a well-known example of military acronym slang.
SNAFU - Wikipedia

FUBAR (Fucked/Fouled Up Beyond All Recognition/Any Repair/All Reason)
List of military slang terms

It is much better to read this page before you do something with rm then after you committed some inexcusable blunder :-)

Many Unix sysadmin horror stories are related to unintended consequences such as side effects of this classic Unix command. Also rm has several idiosyncrasies which are iether easily forgotten or unknown by novice sysadmins.  Usually the disaster strikes due to some unfortunate and unique confluence of factors.  And, of course, 99.999% deletions using rm are successful and do not produce any side effects. This page is about remaining  0.001% or 0.0001% which Anatoly Ivasyuk aptly named "Creative uses of rm" .  They are both very rare and very destructive.

The main problem is that mishaps/SNAFU with rm happen very rarely and people generally do not expect any troubles. Of course, seasoned sysadmins with decades of experience have their set of scars from blunders connected with misused  rm commands, but the problem is that as such incidents happen very rarely (once in a couple of years or even at larger interval), the lessons learned are soon forgotten.  As Benjamin Franklin  quipped that "Experience keeps a dear school, but fools will learn in no other." But this saying is not applicable in this particular case: fool or no fool, if something bad happened three years ago you probably do not remember critical details now, or at the time you write your "one-time" scripts after running of which you discover some strange side effects ;-) In such cases you simply can't learn from experience -- you need specialized and regular safety training.  Please consider rm as dangerous as a sharp blade, or a chainsaw.

You can commit rm blunders and experience corresponding SNAFU both directly on the command line and in your "one-time" scripts (usually shell or Perl scripts). The latter can be even more destructive then the command line blunders because on command line you usually realize that something is deeply wrong in a couple of seconds. In script it can take much longer than that, or not detected until its execution finished. 

One aggravating factor is that rm -r is very fast on modern servers. It is I/O bound so the speed and number of cores does not matter much but presence of SSD greatly enhanced the damage. Even if the resulting list of files and directories is very long (which suspicious in itself and logically would need specifying an additional option like -f to execute without prompt) , rm starts working of it immediately one by one, left to right, unless you specify additional parameters (see -I below). On a server with SSD disks you can easily wipe something between 200GB to one terabyte (depending on the average size of files)  just in several seconds which takes you to realize that the command that you submitted was very wrong.  Around 10-20K files can be wiped in 5 sec -- the time usually needed to realize that submitted command is wrong and cancel it.

As rm actions change data stamp on directories affected by its actions. In case of deeply nested trees with large number of files in deeply nested directories you can see the damage in directories. Damaged directories  with wiped out subdirectories  now have current modification timestamp. Sometimes that to analyze the net result of this wild ride of rm against your data and helps to understand the scope of damage. 

NOTE: it make sense to name directories with valuable data in large data trees Xcritical not Critical so that hopefully in case of SNAFY they were picked up later. 

In any case the only way to keep your "safety training" in rm blunders current is to use your own log and such pages as this one. they should be at the core of self study that you need to repeat at least once a year or before writing script with recognizes important data (for example converts folders in tar.gz files). 

It is much better to read this before you do something with rm then after you committed some inexcusable blunder :-)

One of the key problem here is too simplistic logic of rm command which definitely should  be re-designed in view of 50 year experience of using it in Unix. Some minor steps in this direction were made and  GNU rm does not allow to recursively deleted the root directory. But more can be done. That's why it makes sense to improve this  situation by using a wrapper, and alias it to rm command as those mishaps happened only in interactive sessions. I wrote my own Perl script saferm, while was initially based on safe-rm, but  then completely rewritten for the project were we need to deleted about 8TB of data located in a more then 1K of directories.   

Expansion of basic regular expression .* is one such case (its expansion includes .. which is a parent directory). This specifics of the behavior of rm -r .* can easily be forgotten from one encounter to another, especially if sysadmin works both with Unix and Windows servers. Consequences if executed as root are often tragic... And can cost a week of intensive work to fix.   

Generally, if you have enough space, it is better to move files first to the Trash folder then directly deleting them. 

In cases where there is not spare space to keep the files for a while using proper options also can help to limit the damage.  For example, as of April 2018Linux uses GNU implementation that has relatively new, and very useful option -I, which can replace older option -i as a better solution to the same problems. 

The utility rm  remains pretty Spartan, despite typical "option bloat" of many Linux utilities (see ls as an example). It has just a dozen of options, which each sysadmin should know by heart ;-). Here is the list:

-f, --force

ignore nonexistent files, never prompt

-i

prompt before every removal

-I

prompt once before removing more than three files, or when removing recursively. Less intrusive than -i, while still giving protection against most mistakes

--interactive[=WHEN]

prompt according to WHEN: never, once (-I), or always (-i). Without WHEN, prompt always

--one-file-system

when removing a hierarchy recursively, skip any directory that is on a file system different from that of the corresponding command line argument

--no-preserve-root

do not treat '/' specially

--preserve-root

do not remove '/' (default)

-r, -R, --recursive

remove directories and their contents recursively

-v, --verbose

explain what is being done

--help

display this help and exit

--version

output version information and exit

Two of those are unique to GNU version of rm and  are especially important:

• -I -- prompt once before removing more than three files, or when  removing recursively; less intrusive than -i, while still giving protection against most mistakes. Should be used instead if -i.
• --one-file-system -- do not traverse to other filesystems

NOTE: By default rm -r does not follow symlinks to directories. But if used in  find -exec it does this via find traversing of directories.

Steps that you can do to reduce the chances of the major damage to the data

 Above all, not too much zeal! (Talleyrand’s warning to young diplomats)

Talleyrand

Taleyrand

Wiping out useful data by using rm with wrong parameters or options is probably the most common blunder committed both by novices and experienced sysadmins. The  only difference that experienced sysadmins usually have a backup.

In both case such a mishap happens very rarely, so unless you pay attention and organize for yourself some kind of monthly or quarterly "safety minutes" (which are typically a semi-useless monthly meetings typical for large corporate environment; they can be  made more useful for sysadmins, and IT staff in general,  if  they concentrate on sysadmin horror stores).

Without monthly "safety minutes" you usually forget about the dangers that  rm has and can accidentally step on the same rake again ;-).  As working with any sharp instrument, this danger can't be completely avoided, but in addition to having an up-to-date backup, there are several steps that you can do to reduce the chances of the major damage to the data:

1. If possible, do backup before any dangerous operation. Always do the backup of /etc on the server that you plan to work at the beginning of your working day. This can and probably should be done from dot scripts (such as .bash_profile)  on login.  Especially if you plan to delete files from such system directories such  as /etc,  /boot, /root, etc.  Or if the operation you intend to do is complex and requires traversing filesystem using of find -exec option or find -delete option. That's Rule No. 1
   • If you plan large data reorganization to safe space and do not have full backup you can always use multiple USB drives to create one. they are now are as large as 12 TB each.  Please remember that most painful data loses often happen during such reorganizations.  There is a saying that two data reorganizations have the same effect of one datacenter fire.
2. Use wrapper such as saferm of safe-rm.  It really helps to prevent  many typical cases of accidental deletion of important system and user files.
3. If you can't or do not want to use wrapper and plan to delete multiple files/directories using rm -r or  find -exec (see also Typical Errors In Using Find) please use non-destructive command like ls, tree, or find command   to check what files you are deleting first. Then visually verify correctness of your list.
   • In all more or less complex cases it make sense to delete files/directories as a separate step only after creating and  double checking of the list files to be deleted (sometimes you will be surprised what you will discover in the list created by supposedly "foolprof" regex ;-)  as the input to rm, for example

     ls -r my_super_clever_regex | tee /tmp/files_to_be_deleted
     more tee /tmp/files_to_be_deleted

     xargs -a /tmp/files_to_be_deleted

     or, better (if you have enough  space for files you plan to delete in /Trash or other filesystem)

        xargs -a /tmp/joeuser/my_delete_list -I % mv % /Trash  

   • Using find (with options -exec or -delete) for deletion introduces the whole new class of blunder that you need to be aware of. Please read corresponding page first Typical Errors In Using Find . See also Blunders in using rm with find below.

4. The same is true if you use regular expressions or traverse multiple filesystems. Please run ls, tree, or find command with the argument you intend to use in rm command  and  Wrappers for rm command do it automatically.
5. It is much safer to delete individual files and directories using file manager like WinSCP, or Midnight Commander, then do this directly by typing on the command line.  In this case you have a very important visual feedback, and this less likely to make a blunder. Also you do not need to type the name of file or directory to be deleted, which also help to avoid the whole class of errors.
6. When you are programming a simple script for file deletion, try to type in generated rm command (or any other destructive command) an absolute path to the directory on which you working with at least part of it specified expesidly like in rm /etc/myprogram/confi/*, not as rm $what_i_need_to_delete   In most cases writing commands like  rm $MYDIR is a very bad idea. Using some "safe" prefix is much better deal.  Something like rm /home/joeuser/$MYDIR This protects from cases when your variable for some reason is left undefined or you simply mistyped it pointing to the wrong directory.  One large class of rm blunders is when rm mistakenly operate on root,  or some system directory, or current directory.  Similarly, "chmod -R 755 /Backup/$mypath" is much safer then chmod -R 755 $mybackup
    
7. For small amount of files you can use move to Trash folder instead of deletion, if you have enough space for files that you are deleting if makes sense to delay actual wiping files out as long as possible. You can delete them from the Trash folder later on.
    
   • Many devastating blunders with rm occur when you think that you operate on backup copy of the  system, or important user/application directory, but in reality by mistake operate on a "real thing" or use system directory by  accidentally (and automatically) typing "/" in front of the name of common system directories such as /etc and  /var
   • Even worse the situation happens when you operate on wrong server.
      
   • Moving directories and files to /tmp/Trash folder is always a better deal if you have space.

     del () {
       now="$(date +%Y%m%d_%H%M%S)"
       mkdir -p /tmp/Trash/$now
       mv "$@" /tmp/Trash/$now"
     }

      For additional safety you can write your emptytrash function as:

     emptytrash() {
        rm -rf /tmp/Trash
        mkdir /tmp/Trash  
     }

     There is no way that function can delete anything, but the files you want it to.
      

   • If there are multiple directories with the same name or directories, which you plan to delete, have the same names as system directories (for example /Backup/etc) ,  you can first rename the directory. And only then apply rm -r command to the directory this this new, unique name, for example, /tmp/Junkdir (this way you are less likely mistype the name of the directory or accidentally put slash before its name as in rm /etc  instead of rm etc ...)
      
8. Use option -I instead of -i when deleting multiple files. First of all -i option (which it default alias in Red Hat and other distributions) is really annoying.  Moreover after a while the answer Yes became automatic, ingrained in your brain,  and that's a huge danger.  Many users start eventually simply disable it either by using option -f  or by specifying path, and thus avoiding using this alias. As option  -I is less intrusive it can be recommended as a better replacement. Unfortunately it does not solve that problem of reckless actions, but nothing probably can solve it. It prompts you only once before removing more than three files, or when removing recursively. See rm(1) remove files-directories:

   If the -I or --interactive=once option is given, and there are more than three files or the -r, -R, or --recursive are given, then rm prompts the user for whether to proceed with the entire operation. If the response is not affirmative, the entire command is aborted.

9. To delete subdirectories and files starting with dot you can use "shopt -s dotglob". 
10. Do not improvise complex, potentially destructive commands on the command line without necessity. History of command is your friend. If the command is complex or destructive it makes sense to write in the editor first, not directly improvise it on the command line.  Sometime while typing on command line  you can hit the Enter key or inserts a blank where you do not  intend. Using the editor help to avoid those blunders.
    • Do not improvise creating complex commands involving regular expressions on the command line. This is dangerous and not necessary
    • A weaker version of this rule is to write options and arguments of the command first and after verification add rm (or other dangerous utility name) in front.  
    • Sysadmins  tend automatically type the name of the most common systems directory with slash in front of them  instead of proper relative name (rm -r /etc instead of rm -r etc ) because they are conditioned to type it this way by a long practice and this sequence of keystroke is "etched" in your brain and you do it almost semi-consciously. If you first rename the directory to something else, for example DELETEIT first,  such an error will not occur . A script can check if there is another directory with this name in existence and warn you too.  You can also simply move directory to /Trash folder which has, say 90 day expiration period for files in it

NOTES:

• You also can make the whole /usr directory write-protected by using a separate filesystem for it (/usr/local should be symlinked to /opt where it belongs). There can be some other write protected filesystem as appropriate but usually this is "too little to late" kind of protection.
• You can also use extended attributes for extremely important/vital files.  For example, you can protect symlinks like /lib  and /lib64 in root directory on RHEL7 using expended attributes.
• If you operate server with SElinux in enforcing mode and know SElinux really well, you can use SElinux for file protection.

Use wrapper like saferm

There are two known to the author wrappers for rm:

• saferm -- written by the author and  uses "typed" regular expressions. Types can be l -link, f -- file , d -- directory and  a -- any (no type checking) . There is also type p similar  to a but in which the  comparison is performed using prefix string.  Written  in Perl. Uses tree utility as dependency.  It is oriented on sysadmins who usually know regular expression really well.
• safe-rm -- older and simpler wrapper also written in Perl which uses prefix strings.  Uses prefix strings for protection and as such can be used by users who do not know regular expressions. Also pretty effective

Both need to be  aliased to rm to be more effective. At least for the root.  Aliasing rm to the wrapper is safe because aliases work only in interactive sessions.

Wrapper saferm was written based on horror stories contained in this page and was inspired by safe-rm. It uses a set of Perl regular expression  to protect system directories and user files, which are more powerful and flexible then prefix strings used is safe-rm.  Both are better then nothing. Experience  has  shown that the set of regular expression that you can supply to it in configuration file is pretty powerful tool. It also makes some common sense checks like presence of ".." in the expansion of the arguments.

All key system directories should be protected from accidental deletion, as removal some of them make recovery process more complex. In RHEL7 that list includes also symlinks in the root directory like /lib64. And it goes without saying  that /etc or /usr should be protected better than other directories, with more protected regex defined. 

In addition, using saferm you can block the deletion of directories that are in your private "protected directories and files" list (by default stored in  ~/Saferm.saferm.conf ) .  There are also some indirect methods of checking if the directory you are trying to delete is a wrong directory. For example, if you use level 2 directory /Fav for links some often used directories, you can convert this list into regular expression and that guarantee that those directories will be protected from accidental deletion.

Additional random observations:

• Your home directory and /root often contains important files too and should not be neglected.
• If you use shell variables to specify the target of deletion, the key problem is that some of them can be undefined.  This is especially dangerous if you first cd to the target directory (regex - Safe rm -rf function in shell script - Stack Overflow )

  I've found a big danger with rm in bash is that bash usually doesn't stop for errors. That means that:

  cd $SOMEPATH
  rm -rf *

  Is a very dangerous combination if the change directory fails. A safer way would be:

  cd $SOMEPATH && rm -rf *

  EDIT: @placeybordeaux makes a good point that if $SOMEPATH is undefined or empty cd doesn't treat it as an error and returns 0. In light of that this answer should be considered unsafe unless $SOMEPATH is validated as existing and non-empty first. I believe cd with no args should be an illegal command since at best is performs a no-op and at worse it can lead to unexpected behaviour but it is what it is.

  Instead of cd $SOMEPATH, you should write cd "${SOMEPATH?}". The ${varname?} notation ensures that the expansion fails with a warning-message if the variable is unset or empty (such that the && ... part is never run); the double-quotes ensure that special characters in $SOMEPATH, such as whitespace, don't have undesired effects. – ruakh Jul 13 '18 at 6:46

  There is a set -u bash directive that will cause exit, when uninitialized variable is used. I read about it here, with rm -rf as an example. I think that's what you're looking for. And here is set's manual.

  I would recommend to use realpath(1) and not the command argument directly, so that you can avoid things like /A/B/../ or symbolic links.

Classic blunders with RM command

There are several classic blunders committed using the rm command :

1. Running rm - R command with regular expression, without testing regex expansion first by using ls . You should always use ls -Rl command to test complex rm -R  commands (  -R, --recursive means process  subdirectories recursively).
2. Mistyping complex path or file name in rm commands. It's always better to use copy and paste operation for directories and files used in rm command as it helps to avoid various typos. If you do recursive deletion is useful to have a "dry run" using find or ls to see the files.

   Here are to example of typos:

   rm -rf /tmp/foo/bar/ *

   instead of

   rm -rf /tmp/foo/bar/*

   ===

   Let's assume that you are in directory /backup that has  a subdirectory etc that you want to delete. As path "/etc" is ingrained in sysadmin mind it is very easy automatically/subconsciously type

   rm -r /etc

   instead of

   rm -r etc

   And realize  what you have done in a second or so. That's why for commands which include names OS system directories it is safer to type them in the editor, inspect them and only then execute that command on the command line.  If you are using terminal emulator on Windows desktop then Notepad++ or similar editor is OK.

   This is also why it is prudent to alias rm to the saferm script of your choice, which should prevent such mishaps (aliases are not used in non interactive sessions)

   This actually is an interesting type of "induced" error because /etc is typed daily so often that it kind of ingrained in sysadmin head and can be typed subconsciously instead of etc

3. Using option -r with  .*  regex without understanding its consequences.   That essentially make rm -r recursive and it traverses down to parent directories as this basic regular expression matches ".." (parent directory). 
4. You can accidentally hit Enter by mistake before finishing typing the line containing rm command.  If this is after the first character and the first character is /, or after several characters and the first are the  name of system directory like  /boot in older versions of Linux  you  would find yourself in trouble. Sometimes this happens with the new keyboard when you still do not fully adapted to new keys "feel".
5. You can accidentally insert a blank  after *, this separating the argument into two: "*" and the rest, with corresponding consequences.  Argument * probably should be allowed if and only if it is a single argument on the command  like unless -f is specified.  

Those cases can be viewed as shortcomings of rm implementation (For example, * should be allowed only as a single argument, not as one of several arguments, but Unix expands argument  before passing  them to command  (expansion is done  by shell and there is not simple access to the  original command line) so it is tricky to check; rm also should automatically block deletion of system directories like /,  /etc/ and the list of "protected" directories specified in its config file unless -f  flag is specified. Using has no system attribute so it is difficult to distinguish system files form the rest, but files owned by root:root probably deserve a special treatment as system files, even if one is working as root.

In other works implementation of rm does not take into account complexity of the current Unix environment and its power created an opportunity to wipe our more then you intended in way too many circumstances.

Also Unix does not have system attribute for files although sticky bit on files can be used instead along with ownership of sys instead of root).

Classic blunders in scripts with rm command

The most classic blunder with rm is deletion of wrong directly due to uninitialized variable. For example

TARGET_DIR=/etc
dir_name='Myscript'
cd $TARGET_DIR && rm -r $TARGET_DIR/$DIR_NAME

As you can see the variable used in RM command was types in all caps, but was defined in low case.

Here you need to use what is called defencisve programming. The main idea of which is to introduce addtional "sanity" test before any important operation

For example

TARGET_DIR=/etc
dir_name='Myscript'
if [[ "$TARGET_DIR/$DIR_NAME" = "$TARGET_DIR/" ]] ; then 
   echo [ABNORMAL_SITUATION] Variable $DIR_NAME is not defined
   exit 16
fi   
cd $TARGET_DIR && rm -r $TARGET_DIR/$DIR_NAME

Any script that use rm command but do not have a debug mode is an open  invitation to disaster. 

debug=1
if (( $debug > 0 )) ; then 
  SAFETY='I'
else 
  SAFETY=''
fi    
TARGET_DIR=/etc
dir_name='Myscript'

... ... ...

if [[ -n $TARGET_DIR ]] ; then 
   echo [ABNOMAL SITUATION] Target directory is not defied or is a null string
   exit 128
fi
if [[ "$TARGET_DIR/$DIR_NAME" = "$TARGET_DIR/" ]] ; then 
   echo [ABNORMAL_SITUATION] Variable $DIR_NAME is not defined
   exit 16
fi   
cd $TARGET_DIR && rm -r$SAFETY $TARGET_DIR/$DIR_NAME

It is also very important to check the return code from each command

debug=1
if (( $debug > 0 )) ; then 
  SAFETY='I'
else 
  SAFETY=''
fi    
TARGET_DIR=/etc
dir_name='Myscript'

... ... ...

if [[ -n $TARGET_DIR ]] ; then 
   echo [ABNOMAL SITUATION] Target directory is not defied or is a null string
   exit 128
fi
if [[ "$TARGET_DIR/$DIR_NAME" = "$TARGET_DIR/" ]] ; then 
   echo [ABNORMAL_SITUATION] Variable $DIR_NAME is not defined
   exit 16
fi   
cd $TARGET_DIR && rm -r$SAFETY $TARGET_DIR/$DIR_NAME

if (( $? > 0 )) ; then 
   echo "Abnormal return code from rm $?
fi

See also shell - How to check if a variable is set in Bash - Stack Overflow

Generalized approach to checking important parameters in subroutines and before passing them to external programs like rm is called defensive programming.unfortunatly Wipipedia article is exremly weak, but some ideas applicable for bash can be derived from articles devotes to defensive programming in Java. For example  Defensive programming the good, the bad and the ugly - Enterprise Craftsmanship

Checks above are example of assertions about the value of parameters. 

The second common blunder in using rm command in scripts is absence of checking of return code of command that prepare rm environment (if such commands are used)

For example

cd /Etc/mc; rm *.conf

It is always safer to check the return code. For all Unix command return code 0 means success. So classic rewrite of the command above will be

cd /Etc/mc && rm *.conf

cd Etc
if (( $? == 0 )) ; then 
   rm *.conf
   if (( $? > 0 )) ; then 
      echo [FAILURE] rm return code is $?
   fi
fi

cd Etc
if (( $? == 0 )) ; then 
   rm *.conf
   if (( $? > 0 )) ; then 
      echo [FAILURE] rm return code is $?
   fi
fi

Unfortunately not all GNU utilities document return code in man pages. You need to experiment  is such case to determines which situation result in which return code.

NOTE: The examples above are presented just for the illustration of the concept. Generally you should avoid executing rm command on the current directory in your scripts, because that increases possibility to execute the command in the current directory which is the one you intend to use.  Always provide and verify the directory as a parameter to rm

Backup files before executing any "large scale" rm command, especially if you deleting files in system directory or an important files in use of data directories and the total size is not that large

"Make back of /etc first"  is the Rule No.1 if you delete something  from /etc directory or other system directories (/boot, /root, etc). Generally you should create the tarball of /etc directory at the beginning of the first root session for the particular day. Making tarball of the /etc directory on modern servers takes a couple of seconds.

Backups of subtrees affected before execution of rm command are also important. can save quite a lot of nerves in situations that otherwise can s can be devastating. For example, in the example above you would erase all your files and subdirectories in the /etc directory. Modern flavors of Unix usually prevent erasing /,  but not /etc. I think Red hat version  of GNU rm prevents erasing all level 2 system directories, but I am not sure.

Any attempt to remove a file whose last file name component is . or .. is rejected without any prompting, as mandated by POSIX.

If we are talking about more then 1GB of files, you can also move directories to /Trash folder and delete them after, say 7 days or so.  Later you can delete files that are say, older then 7 days using a cron script, or you can delay deletion until the total size of this directory exceeds a certain threshold. 

Within 10TB limit, You can always buy USB 3.0 drive and use it for such operations. Even if you buy it on your own money it is much cheaper than a couple of sleepless nights trying to recover lost information. 

There no excuse not to provide a backup in case of any large scale deletion.

More on blunders with the deleting a directory from the backup

Experience keeps a dear school, but fools will learn in no other. ~Benjamin Franklin

The saying is that experience keeps the most expensive school but fools are unable to learn in any other ;-).

It is dangerous to type destructive command directly on the command line as there are some idiosyncrasies that can play with you a very bad joke. Often backup has directories with the same name as systems directories simply because this is a backup of system directories. I once automatically typed /etc instead of etc trying to delete directory /backup/etc or something like that  to free space on a backup directory on a production server (/etc probably in engraved in sysadmin head as it is typed so often and can be substituted for etc subconsciously).  I realized that it was mistake and cancelled the command, but it was a fast server and one third of /etc was gone. The rest of the day was spoiled...  Actually not completely: I learned quite a bit about the behavior of AIX in this situation and the structure of AIX /etc directory this day, so each such disaster is actually a great learning experience, almost like one day training course ;-).

But it's much less nerve wracking to get this knowledge from the course... Another interesting thing is having backup was not enough is this case -- enterprise backup software stopped working on a damaged server. The same was true for telnet and ssh. And this was a remote server in a datacenter across the country.  I restored the directory on the other non-production server (first overwriting its /etc directory in this second box with the help of operations, tell me about cascading errors and Murphy law :-). In this case one ssh  session survives and helped to transfer the tar file.  As netcat does not requires authentication and if it is statically linked  it might help in such cases as you might be able to transfer missing files using it.

Such blunders are really persistent as often used directories are types in a "semiconscious" fashion, in "autopilot" mode and you realize the blunder only after you hit Enter. 

In such cases network services with authentication stop working the only way to transfer files is using CD/DVD, USB drive (if cp works, which in RHEL is not given as it is dynamically linked) or netcat. That's why it is useful to have netcat on servers: netcat is the last resort file transfer program when  services with authentication like ftp or scp stop working.  It is especially useful to have it if the datacenter is remote.

A simple extra space often produced horrible consequences:

cd /usr/lib
ls /tmp/foo/bar

I typed

rm -rf /tmp/foo/bar/ *

instead of

rm -rf /tmp/foo/bar/*

The system doesn’t run very will without all of it’s libraries……

NOTE: You can block such behavior requiring that if -r option was given you should have one and only one argument to rm. That probably should be a part of functionality of your "saferm" script.

Important class of subtle Unix errors: dot-star errors

A popular class of recursive rm errors are so called dot-star-errors. They happen when rm is using with -r option or when rm is used  in find -exec option.  Novice sysadmins usually do not realize that '.*' also matches '..' often with disastrous consequences.  If you are in any doubt as for how a wildcard will expand use echo command to see the result.

• From: [email protected] (Contractor Bob Johnson)
  Organization: Tinker Air Force Base, Oklahoma

  Another horror story (mine this time)...

  Cleaning out an old directory, I did 'rm *', then noticed several files that began with dot (.profile, etc) still there.  So, in a fit of obtuse brilliance, I typed...

      rm -rf .* &

  By the time I got it stopped, it had chewed through 3 filesystems which all had to be restored from tape (.* expands to ../*, and the -r makes it keep walking up the directory tree).  Live and learn...
   

• My mistake on SunOS (with OpenWindows) was to try and clean up all the '.*' directories in /tmp. Obviously "rm -rf /tmp/*" missed these, so I was very careful and made sure I was in /tmp and then executed

  "rm -rf ./.*".

  I will never do this again. If I am in any doubt as to how a wildcard will expand I will echo it first.

Using "convenience" symlinks to other (typically deeply nested) directories inside a directories and forgetting about them

Command  rm does not follows symlinks but find does.  So if you have a symlink to some system on important directory from the directory you are deleting using

find -exec rm {} \;

you are up to nasty surprise

That's the main danger of "convenience symlinks" which are used to simplify access to deeply nested directories.  They are more dangerous then useful. Using aliases or functions is a better deal.

If you use them, do this only from the level two directory created specially for this purpose like /Fav and protect this directory in your saferm script.

Disastrous typos in name of the directory or regex -- unintended space

There are some exotic typos that can lean you to troubles, especially with -r  option. One of them is unintended space:

 rm /home/joeuser/old *

instead of

rm /home/joeuser/old*

Dumb expression... Gradual realization of what you have done. Cold sweat..

Wrapper can catch some of those errors if  /home/joeuser/old does not exists.

Or, similarly:

 rm * _old

instead of

rm *_old

Such a mistake is even more damaging  is you use -r option. For example:

rm -rf /tmp/foo/bar/ *

instead of

rm -rf /tmp/foo/bar/*

So it is prudent to block execution of rm commands with multiple argument your saferm script, if option  -r is specified.

Root deletion protection

To remove a file you must have write permission on the folder where it is stored. Sun introduced "rm -rf /" protection in Solaris 10, first released in 2005. Upon executing the command, the system now reports that the removal of / is not allowed.

Shortly after, the same functionality was introduced into FreeBSD version of rm  utility.

GNU rm  refuses to execute rm -rf /  if the --preserve-root  option is given, which has been the default since version 6.4 of GNU Core Utilities was released in 2006.

No such protection exists for critical system directories like /etc /bin, and so on but you can imitate it putting file "-i" into such directories or using a wrapper for the interactive usage of the command. 

Using echo command to see expansion of your wildcards, if any

To understand what will be actually being executed after shell expansion, preface your rm command with echo. For example, if there is a filename starting with the dash you will receive very confusing message from rm

	$ rm *
	rm: unknown option -- -
	usage: rm [-f|-i] [-dPRrvW] file ...

That side effect can be used for protection of system directories. To find out what caused it prefix the command with echo

echo rm *

You can generate list of files dynamically for rm (piping it via xargs if it is too large.

But a better way is to include  generation and analysis of the list of files be  deleted into your saferm script and switch to generation of the list instead of actual deletion is any "suspicious files or circumstances are detected.

alias rm="saferm"

Please note that alias used in RHEL (alias rm="rm -i"), unfortunately, encourages a tendency to mindlessly pound y  and the return key to affirm removes - until you realize that got just past the file you need to keep.

Using a file starting with - as stoppers for unintended use of rm. Undeletable files

The rm command accepts the -- option which will cause it to stop processing flag options from that point forward. This allows the removal of file names that begin with a dash (-).

rm -- -filename

Actually accidental creation of files starting with dash is a common problem for novice sysadmin.  The problem is that if I type rm -foo, the rm command treats the filename as an option.

There are two simple ways to delete such a file. One was shown above and based on supplying an empty option argument. The second to use a relative or absolute pathname including at least one directory in  front:

rm ./--protected

rm /home/user/--protected

Here the debugging trick that we mentioned above -- prefacing rm command with echo really helps to understand what happens.

Deletion of files with non-printable characters often lead to unintended consequences

The safest way to do this is to use inodes for such files. You can also specify regular expression replacing non-printable characters in the name with ?  for each bad character. For example:

rm bad?file?name

But trying " a shortcut" which does not require meticulously putting ? in the name like" 

rm bad* 

Another way to delete files that have control characters in the name is to  use  -i option and an asterisk, which gives you the option of removing each file in the directory — even the ones that you can't type.

% rm -i *
rm: remove faq.html (y/n)? n
rm: remove foo (y/n)? y
%

The -i option may also be helpful when you are dealing with files with Unicode characters that appear to be regular letters in your locale, but that don't match patterns or names you use otherwise.

You can also create the function for deletion of files with special characters. Something along the lines:

# cat  /root/del_special_characters.sh
   if [ ! -d /tmp/Special_files ] ; then
      mkdir /tmp/Special_files
   fi
   genfile=/tmp/Special_files/list`date +"%y%m%d"`
   echo dev/null > $genfile
   find . -maxdepth 1  -type f -ls | fgrep 'a' | while  read myline
   do
      echo Processing: $myline
      IFS=' ' # Warning: the IFS variable means split by one of characters in a set
      set $myline
      inode=$1
      echo inode=$inode
      new_name=${!#};
      new_name=`tr 'a' '_' <<< "$new_name"`
      echo "new_name=$new_name"
      echo mv $inode $new_name
      echo mv $inode $new_name >> $genfile
   done
   echo "Generated commands:"
   cat $genfile

A great way to discover files with control characters in them is to use the -q option to the Unix ls command (some systems also support a useful -b option). You can, for example, alias the ls command:

alias lsq='ls -liq '

Files that have control characters in their filenames will then appear with question marks:

lqs f*
faq.html                fo?o

Blunders in using rm with find

This is class or blunders that deserves a page of its own. See Typical Errors In Using Find  for more information about this SNAFU

You need to be extra careful running rm in an -exec option of find command. There are many horror stories in which people delete large parts of their filesystem due to typos of sloppy find command parameters. Again it is prudent first to test it using ls command to see the actual list of files to be deleted. Sometimes it is quite different from what you intended to do :-).

you can also generate the list using ls command, inspect it in a separate step. And only then pipe it into rm :

cat rm.lst | xargs -t -n1 rm 

Dr. Nikolai Bezroukov

Recommended Links

Softpanorama Recommended

• bash - Safe Directory Delete - Unix & Linux Stack Exchange
• Cleaning Up Your -tmp...The Safe Way Issue 18
• Is it safe to delete stuff in /tmp (SUSE 10.2)
• delete - Can I safely remove all the files in /tmp - Ask Ubuntu
• linux - Delete all of -var-log - Server Fault

Saferm implementations

• safe-rm

  Excerpt from the web site:

  Safe-rm is a safety tool intended to prevent the accidental deletion of important files by replacing /bin/rm with a wrapper, which checks the given arguments against a configurable blacklist of files and directories that should never be removed.

  Users who attempt to delete one of these protected files or directories will not be able to do so and will be shown a warning message instead:

  $ rm -rf /usr
  Skipping /usr

• How to install safe-rm on Debian Unstable (Sid) – Kreation Next – Support
• saferm Safely remove files, moving them to GNOME/KDE trash instead of deleting by Eemil Lagerspetz
• shell - Make `rm` move to trash - Unix & Linux Stack Exchange
• linux - How do I prevent accidental rm -rf - - Server Fault

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: September 04, 2019