Softpanorama

May the source be with you, but remember the KISS principle ;-)
Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
(slightly skeptical) Educational society promoting "Back to basics" movement against IT overcomplexity and  bastardization of classic Unix

Cron and Crontab commands (Vixie Cron)

News Job schedulers Recommended Links Reference

/etc/crontab

crontab command
Crontab structure cron.allow cron.deny Crontab specification tips at  command batch command
Macros Ordinary users in SLES 11 and 10 are denied access to cron logrotate Grid engine tmpwatch Parallel command execution
Troubleshooting Perl Admin Tools Tips Sysadmin Horror Stories

Humor

Etc

Introduction

Cron is a standard scheduler for Unix servers. There are several cron implementations. All follow the principle of fire & forget. They have no ability to monitor jobs success/failures and to provide dependency-based scheduling. The only thing they can do is to mail the output to the user under which job was invoked (typically root). Along with calendar based invocation, cron has basic batch facilities (see  batch command ), but  grid engine which is also open source component of Unix/Linux provides much more sophisticated and powerful batch environment. Batch command is actually an example of dependency based scheduling as command is executed when load of the server falls below specified for atd daemon threshold (1.5 by default). There are several queues that you can populate and each queue can be populated with multiple jobs which will be executed sequentially, one after another.  See batch command for more details.  

Absence of dependency-based scheduling is less of a limitation then one might think. A simple status-based dependency mechanism is easy to implement in shell. In this case the start of each job can depend of existence of a particular "status-file" that was created (or not, in case of failure) during some previous step. Those status-files can survive reboot or crash of the server.

With at command the same idea can be implemented by cancelling the execution of next scheduled step from a central location using ssh . In this case ssh can serve as central scheduling daemon and at daemon as local agent. With the universal adoption of ssh daemon remote scheduling is as easy as local. For example, if a backup job fails it did not create the success file, then each job dependent on it should checks the existence of the file and exit if the file is not found. In more general way one can implement script "envelope" -- a special script that creates status file and send messages at the beginning and at the end of each step to the monitoring system of your choice. Using at commands allow you to cancel or move to a different time all at commands dependent on successful completion of the backup.

Another weakness of cron is that its calendaring function is not sophisticated enough to understand national holidays, furlough, closure, maintenance periods, plants shut downs, etc. Avoiding running workload on holidays or specific days (e.g. an inventory day) is relatively easy to implement via standard envelope which performs such checks. Additionally you can store all the commands with parameters in /root/cronrun.d or similar directory and run checks within them via functions or external scripts. You can also specify in cron command  /usr/local/bin/run with appropriate parameters (the first parameter is the name of script command to run, everything parameters passed to the script). For example:

@dayly /usr/local/bin/run tar cvzf /var/bak/etc`date%y%m%d`.tgz
@daily /root/cronrun/backup_image

You can also prefix command with the "checking script" (let's name in canrun) script,  and exit, if it returns "false" value. For example

@daily /usr/local/bin/run && tar cvzf /var/bak/etc`date%y%m%d`.tgz

Another, more flexible, but slightly more complex way way is to use the concept of "scheduling day" with particular start time and length (in some cases it is beneficial to limit length of scheduling day to just one shift or extend it to 48 hours period). In this case the "start of scheduling day" special script filters a sequence of at commands that is scheduled that day and then propagates this script to all servers that need such commands via ssh or some  parallel command execution command.  In a simplest case you can generate such a sequence using  cron itself, when all running command are prefixed by echo command that generated at command with the appropriate parameter and redirects output to "the schedule of the day" file.   For example

@hourly echo 'echo "uptime" | at' `date +%H:%M` >> /root/schedule_of_the_day

This allows to centralize all such the checks on a single server -- mothership.

After this one time "feed" servers become autonomous and execute the generated sequence of jobs, providing built-in redundancy mechanism based on independence of local cron/at daemons from the scheduler on the "mothership" node. Failure of the central server (aka mothership) does not affect execution of jobs of satellite servers until the current scheduling day ends.

Due to day light saving time changes it is prudent not to schedule anything important between midnight and 3 AM.

Due to day light saving time changes it is prudent not to schedule anything important between midnight and 3 AM.

Linux and OpenSolaris use Vixie cron which is somewhat richer in facilities than traditional SysV cron daemon. Two important extensions are slash notation for specified period ("*/5" means each five minutes) and "@" keywords such as @reboot which allow to specify scripts that will be executed after reboot. This is a simpler way then populating /etc/init.d/local script which serves for the same purpose.

Without those two extensions Vixie cron is a compatible with SysV cron.

To determine if Vixie cron is installed, use the rpm -q cron command on Suse 10 and rpm -qi vixie-cron on Red Hat.

To determine if the service is running, your can use the command /sbin/service crond status. Cron is one of the few daemons that does not require restart when the configuration changed via crontab command.

Vixie cron extensions include:

@reboot echo `hostname` was rebooted at `date` | mail -s "Reboot notification" [email protected]

Other macros include:

Other features of Vixie cron are pretty standard.

Linux distributions make cron somewhat more complex and flexible by splitting crontab into several include file with predefined names cron.daily, cron.hourly,  cron.weekly and cron.monthly that are executed from the "central" crontab. 

  1. /etc/crontab (Red Hat & Suse10) -- the master crontab file which like /etc/profile is always executed first and contains several important settings including shell to be used as well as invocation of the script which process predefined five directories:

    Implementations of this feature are different in Red Hat and Suse 10. See below for details of each implementation.

  2. etc/cron.allow - list of users for which cron is allowed.  The files "allow" and "deny" can be used to control access to the "crontab" command (which serves for listing and editing of crontabs; direct access to them is discouraged). Cron does not need to be restarted of send HUP signal to reread those files.
     
  3. /etc/cron.deny - list of users for which cron is denied. Note: if both cron.allow and cron.deny files exist the cron.deny is ignored.
     
  4. crontab files: there are multiple (one per user) crontab files which list tasks and their invocation conditions. All crontab files are stored in a read-protected directory, typically /var/cron).  Those files is not edited directly. There is a special command crontab which serves for listing and editing them.

/etc/crontab

In both Suse and Red Hat there is a master crontab file /etc/crontab which like /etc/profile is always executed first (or to be more correct is a hidden prefix of  "root" crontab file).

By default it contains several settings and invocation of script which runs each 15 min) and processes four predefined directories (which represent "Linux extension" of standard cron functionality).

Here is example from Suse:

SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * *   root  test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/null 2>&1

Notes:

As we already mentioned, details of implementation are different for Red Hat and Suse.

Red Hat implementation of /etc/crontab

In Red Hat the master crontab file /etc/crontab uses /usr/bin/run-parts script to execute content of predefined directories. It contains the following lines

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly

Bash script run-parts contains for loop which executes all components of corresponding directory one by one.  This is a pretty simple script. You can modify if you wish:

#!/bin/bash
# run-parts - concept taken from Debian
# keep going when something fails
set +e

if [ $# -lt 1 ]; then
        echo "Usage: run-parts <dir>"
        exit 1
fi

if [ ! -d $1 ]; then
        echo "Not a directory: $1"
        exit 1
fi

# Ignore *~ and *, scripts
for i in $1/*[^~,] ; do
        [ -d $i ] && continue
        # Don't run *.{rpmsave,rpmorig,rpmnew,swp} scripts
        [ "${i%.rpmsave}" != "${i}" ] && continue
        [ "${i%.rpmorig}" != "${i}" ] && continue
        [ "${i%.rpmnew}" != "${i}" ] && continue
        [ "${i%.swp}" != "${i}" ] && continue
        [ "${i%,v}" != "${i}" ] && continue

        if [ -x $i ]; then
                $i 2>&1 | awk -v "progname=$i" \
                              'progname {
                                   print progname ":\n"
                                   progname="";
                               }
                               { print; }'
        fi
done
exit 0

This extension allows adding cronjobs by simply writing a file containing an invocation line into appropriate directory.  For example, by default /etc/cron.daily directory contains:

# ll
total 48
-rwxr-xr-x 1 root root  379 Dec 18  2006 0anacron
lrwxrwxrwx 1 root root   39 Jul 24  2012 0logwatch -> /usr/share/logwatch/scripts/logwatch.pl
-rwxr-xr-x 1 root root  118 Jan 18  2012 cups
-rwxr-xr-x 1 root root  180 Mar 30  2011 logrotate
-rwxr-xr-x 1 root root  418 Mar 17  2011 makewhatis.cron
-rwxr-xr-x 1 root root  137 Mar 17  2009 mlocate.cron
-rwxr-xr-x 1 root root 2181 Jun 21  2006 prelink
-rwxr-xr-x 1 root root  296 Feb 29  2012 rpm
-rwxr-xr-x 1 root root  354 Aug  7  2010 tmpwatch
You can study those scripts to add your own fragments or rewrite them completely.

Invocation of tmpwatch from /etc/cron.daily

Another "standard" component of /etc/cron.daily  that deserves attention is a cleaning script for standard /tmp locations called tmpwatch:

cat tmpwatch
flags=-umc
/usr/sbin/tmpwatch "$flags" -x /tmp/.X11-unix -x /tmp/.XIM-unix \
        -x /tmp/.font-unix -x /tmp/.ICE-unix -x /tmp/.Test-unix \
        -X '/tmp/hsperfdata_*' 240 /tmp
/usr/sbin/tmpwatch "$flags" 720 /var/tmp
for d in /var/{cache/man,catman}/{cat?,X11R6/cat?,local/cat?}; do
    if [ -d "$d" ]; then
        /usr/sbin/tmpwatch "$flags" -f 720 "$d"
    fi
done
The utility can is also invoked from several other scripts. For example cups script provides for removal of temp files from /var/spool/cups/tmp tree using /usr/sbin/tmpwatch utility
#!/bin/sh
for d in /var/spool/cups/tmp
do
    if [ -d "$d" ]; then
        /usr/sbin/tmpwatch -f 720 "$d"
    fi
done
exit 0
If you do not use caps it is redundant, and you can rename it to ~cups to exclude from execution. You can also use it as a prototype for creating your own script(s) to clean temp directories of applications that you do have on the server.

Suse implementation of /etc/crontab

Suse 10 and 11 have identical implementations of this facility.

There is also "master" crontab at /etc/crontab that like /etc/profile is always executed. But details and the script used (/usr/lib/cron/run-crons) are different from Red Hat implementation:

SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * *   root  test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/nul
l 2>&1

Suse also uses the same five directories as Red Hat. Each of those can contain scripts which will be executed by shell script /usr/lib/cron/run-crons. The latter is invoked each 15 minutes like in Red Hat.  But the  script itself is quite different, and  as is typical for Suse much more complex then one used by Red Hat. To make things more confusing for sysadmins, at the beginning of its execution it sources the additional file /etc/sysconfig/cron which contains additional settings which control the script (and by extension cron) behavior:

/home/bezroun # cat /etc/sysconfig/cron
## Path:        System/Cron/Man
## Description: cron configuration for man utility
## Type:        yesno
## Default:     yes
#
# Should mandb and whatis be recreated by cron.daily ("yes" or "no")
#
REINIT_MANDB="yes"

## Type:        yesno
## Default:     yes
#
# Should old preformatted man pages (in /var/cache/man) be deleted? (yes/no)
#
DELETE_OLD_CATMAN="yes"

## Type:        integer
## Default:     7
#
# How long should old preformatted man pages be kept before deletion? (days)
#
CATMAN_ATIME="7"
## Path:        System/Cron
## Description: days to keep old files in tmp-dirs, 0 to disable
## Type:        integer
## Default:     0
## Config:
#
# cron.daily can check for old files in tmp-dirs. It will delete all files
# not accessed for more than MAX_DAYS_IN_TMP. If MAX_DAYS_IN_TMP is not set
# or set to 0, this feature will be disabled.
#
MAX_DAYS_IN_TMP="0"

## Type:        integer
## Default:     0
#
# see MAX_DAYS_IN_TMP. This allows to specify another frequency for
# a second set of directories.
#
MAX_DAYS_IN_LONG_TMP="0"

## Type:        string
## Default:     "/tmp"
#
# This variable contains a list of directories, in which old files are to
# be searched and deleted. The frequency is determined by MAX_DAYS_IN_TMP
#
TMP_DIRS_TO_CLEAR="/tmp"

## Type:        string
## Default:     ""
#
# This variable contains a list of directories, in which old files are to
# be searched and deleted. The frequency is determined by MAX_DAYS_IN_LONG_TMP
# If cleaning of /var/tmp is wanted add it here.
#
LONG_TMP_DIRS_TO_CLEAR=""

## Type:        string
## Default:     root
#
# In OWNER_TO_KEEP_IN_TMP, you can specify, whose files shall not be deleted.
#
OWNER_TO_KEEP_IN_TMP="root"

## Type:        string
## Default:     no
#
# "Set this to "yes" to entirely remove (rm -rf) all  files and subdirectories
# from the temporary directories defined in TMP_DIRS_TO_CLEAR on bootup.
# Please note, that this feature ignores OWNER_TO_KEEP_IN_TMP - all files will
# be removed without exception."
#
# If this is set to a list of directories (i.e. starts with a "/"), these
# directories will be cleared instead of those listed in TMP_DIRS_TO_CLEAR.
# This can be used to clear directories at boot as well as clearing unused
# files out of other directories.
#
CLEAR_TMP_DIRS_AT_BOOTUP="no"

## Type:         string
## Default:      ""
#
# At which time cron.daily should start. Default is 15 minutes after booting
# the system. Example setting would be "14:00".
# Due to the fact that cron script runs only every 15 minutes,
# it will only run on xx:00, xx:15, xx:30, xx:45, not at the accurate time
# you set.
DAILY_TIME=""

## Type:         integer
## Default:      5
#
# Maximum days not running when using a fixed time set in DAILY_TIME.
# 0 to skip this. This is for users who will power off their system.
#
# There is a fixed max. of 14 days set,  if you want to override this
# change MAX_NOT_RUN_FORCE in /usr/lib/cron/run-crons
MAX_NOT_RUN="5"

## Type:        yesno
## Default:     no
#
# send status email even if all scripts in
# cron.{hourly,daily,weekly,monthly}
# returned without error? (yes/no)
#
SEND_MAIL_ON_NO_ERROR="no"

## Type:        yesno
## Default:     no
#
# generate syslog message for all scripts in
# cron.{hourly,daily,weekly,monthly}
# even if they haven't returned an error? (yes/no)
#
SYSLOG_ON_NO_ERROR="yes"

## Type:       yesno
## Default:    no
#
# send email containing output from all successful jobs in
# cron.{hourly,daily,weekly,monthly}. Output from failed
# jobs is always sent. If SEND_MAIL_ON_NO_ERROR is yes, this
# setting is ignored.  (yes/no)
#
SEND_OUTPUT_ON_NO_ERROR="no"

From the content of the script you can see, for example, that time of execution of scripts in /etc/cron.daily is controlled by env. variable $DAILY_TIME which can be set in /etc/sysconfig/cron system file. This way you can run daily jobs at time slot different from  hourly, weekly and monthly jobs. 

This is an example of unnecessary complexity dictated by desire to create a more flexible environment, but which   just confuses most sysadmin, which neither appreciate, no use provided facilities.  

As you can see from the content of the file listed above, considerable attention in /etc/sysconfig/cron is devoted to the problem of deletion of temp files. Which is an interesting problem in a sense that you can not just delete temporary files on running system. Still you can use an init task that runs tmpwatch utility before any application daemons are started.

Editing cron files with crontab command

NOTE: Creating a backup file before any non-trivial crontab modification is a must.

To list and edit cron file one should use crontab command which copies the specified file or standard input if no file is specified, into a directory that holds all users' crontabs. It can also invoke the editor with option -e to edit exiting crontab.

crontab Command Switches

If option -u is not specified crontab operates with cron files for the current user.

There are two man pages for crontab. You can view them via WWW browser from die.net:

Users are permitted to use crontab, if their names appear in the file /etc/cron.allow. If that file does not exist, the file /etc/cron.deny is checked to determine if the user should be denied access to crontab. If neither file exists, only a process with appropriate privileges is allowed to submit a job. If only /etc/cron.deny exists and is empty, global usage is permitted. The cron.allow and cron.deny files consist of one user name per line.

See Controlling access to cron for more info.

Crontab structure

A crontab file can contain two types of instructions to the cron daemon:

Each user has their own crontab, and commands in any given crontab will be executed as the user who owns the crontab.

Blank lines and leading spaces and tabs are ignored. Lines whose first non-space character is a pound-sign (#) are comments, and are ignored. Note that comments are not allowed on the same line as cron commands, since they will be taken to be part of the command. Similarly, comments are not allowed on the same line as environment variable settings.

An active line in a crontab can be either an environment setting line or a cron command line.

  1. An environment setting should be in the form, name = value .  The spaces around the equal-sign (=) are optional, and any subsequent non-leading spaces in value will be interpreted as the value assigned to name.

    The value string may be placed in quotes (single or double, but matching) to preserve leading or trailing blanks. The name string may also be placed in quote (single or double, but matching).

    NOTE: Several environment variables should be set via /etc/crontab which we discussed above (SHELL, PATH, MAILTO). Actually in view of /etc/crontab existence, setting environment variables via cron looks redundant and should be avoided.
     

  2. Each cron command is a single line that consists of six fields. One line of cron table specifies one cron job. A cron job is a specific task that runs a certain number of times per minute, day, week, or month. For example, you can use a cron job to automate a daily MySQL database backup. The main problem with cron jobs is that if they aren't properly configured and all start at the same time they can cause high server loads. So it is prudent to have different time for start of hour and daily jobs. In case of important jobs which run once a day it makes sense to configure your cron job so that the results of running the scheduled script are emailed to you, not to root under which the job was run.

    There are two main ways by which you create a cron job. On is using your Web administration panel ( most *nix webhosting providers offer Web-base interface to cron) or using shell access to your server.

    A crontab expression is a string comprising 6 or 7 (with year) fields separated by white space. Its structure has the structure shown in the table:

    Field Meaning Allowed range Allowed special characters Example
    1 Minutes that have to pass after the selected hour in order to execute the task 0-59 * / , - 30, which means 30 minutes after the selected hour
    2 Hours at which the task has to be executed 0-23 * / , - 04, which means at 4 O'clock in the morning
    3 Days of the month on which this task has to be executed 1-31 * / , -  *, which means that every day of the selected month
    4 Months during which the task has to be executed 1-12 * / , - 3-5, which means run the task in the months of March, April & May First 3 letters of the Month name. Case doesn't matter. E.g. Jan
    5 Days of the week on which this task has to be run 0-7 * / , -  * means all days of the selected weeks Numeric value or first 3 letters of the Day name. Case doesn't matter (Sun or sun)
    (0 or 7 is Sun, 1 is Mon...)
    6 Name of the program (task) to be executed Any program

    %

    absolute path to executable required

Special characters

Support for each special character depends on specific distributions and versions of cron

Asterisk ( * )
The asterisk indicates that the cron expression will match for all values of the field; e.g., using an asterisk in the 4th field (month) would indicate every month.
Slash ( / )
Slashes are used to describe increments of ranges. For example 3-59/15 in the 1st field (minutes) would indicate the 3rd minute of the hour and every 15 minutes thereafter. The form "*/..." is equivalent to the form "first-last/...", that is, an increment over the largest possible range of the field.
Percent ( % )
Percent-signs (%) in the command, unless escaped with backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input.
Comma ( , )
Commas are used to separate items of a list. For example, using "MON,WED,FRI" in the 5th field (day of week) would mean Mondays, Wednesdays and Fridays.
Hyphen ( - )
Hyphens are used to define ranges. For example, 2000-2010 would indicate every year between 2000 and 2010 CE inclusive.

Crontab specification tips

The last field (the rest of the line) specifies the command to be run. The entire command portion of the line, up to a newline or % character, will be executed by /bin/sh or by the shell specified in the SHELL variable of the /etc/crontab

Percent-signs (%) in the command, unless escaped with backslash (\), will be changed into newline characters, and all data after the first % will be sent to the command as standard input.

Note: The day of a command's execution can be specified by two fields --- day of month, and day of week. If both fields are restricted (i.e, aren't *), the command will be run when either field matches the current time. For example, "30 4 1,15 * 5'' would cause a command to be run at 4:30 am on the 1st and 15th of each month, plus every Friday.

Recommended header for crontab

To avoid mistakes it is recommended to include the following header in the crontab

# minute (0-59),
# |      hour (0-23),
# |      |       day of the month (1-31),
# |      |       |       month of the year (1-12),
# |      |       |       |       day of the week (0-6 with 0=Sunday, 1-Monday).
# |      |       |       |       |       commands

Macros

Instead of the first five fields, in vixie-cron you can use one of eight predefined macros:

string meaning
------ -------
@reboot Run once, at startup.
@yearly Run once a year, "0 0 1 1 *".
@annually (same as @yearly)
@monthly Run once a month, "0 0 1 * *".
@weekly Run once a week, "0 0 * * 0".
@daily Run once a day, "0 0 * * *".
@midnight (same as @daily)
@hourly Run once an hour, "0 * * * *".

The most useful is @reboot which does provide new functionality. Usefulness of others macros is questionable.

Note about daytime savings

If you are in one of the countries that observe Daylight Savings Time, jobs scheduled during the rollback or advance will be affected. In general, it is not a good idea to schedule jobs from 1pm to 3pm.

For US timezones (except parts of IN, AZ, and HI) the time shift occurs at 2AM local time. For others, the output of the zdump(8) program's verbose (-v) option can be used to determine the moment of time shift.

Controlling access to cron

Cron has a built in feature of allowing you to specify who may, and who may not use it. It does this by the use of /etc/cron.allow and /etc/cron.deny files:

  1. /etc/cron.allow - list of users for which cron is allowed.  The files /etc/cron.allow  and /etc/cron.deny can be used to control access to the "crontab" command (which serves for listing and editing of crontabs; direct access to them is discouraged). Cron does not need to be restarted of send HUP signal to reread those files.
  2. /etc/cron.deny - list of users for which cron is denied.

Note: if both cron.allow and cron.deny files exist the cron.deny is ignored.

 If you wanted that only selected users can use cron, you could add the line ALL to the cron.deny file and put the list of those users into cron.allow file:

echo ALL >>/etc/cron.deny
If you want user apache to be able to use cron you need to add the appropriate line to /etc/cron.allow file. For example:
echo apache >>/etc/cron.allow

If there is neither a cron.allow nor a cron.deny file, then the use of cron is unrestricted (i.e. every user can use it). If you put a name (or several names) into cron.allow file, without creating a cron.deny file, it would have the same effect as creating a cron.deny file with ALL in it. This means that any subsequent users that require cron access should be put in to the cron.allow file.

For more information about cron.allow and cron.deny file see Reference item cron.allow and cron.deny

Output from cron

By default the output from cron gets mailed to the person specified in the MAILTO variable. If this variable is not defined it is mailed to the owner of the process.  If you want to mail the output to someone else, you can just pipe the output to the command mailx command. For example:
echo test | mail -s "Test of mail from cron" [email protected]

If you have a command that is run often, and you don't want to be emailed the output every time, you can redirect the output to a log file (or /dev/null, if you really don't want the output).  For example

cmd >> log.file

Now you can create a separate cron job that analyses the log file and mail you only if this is an important message. Or just once a day, if the script is not crucial. This way you also can organize log rotation.  See logrotate for details

Backup and restore

The loss of crontab is a serious trouble. This is one of a typical sysadmin blunders (Crontab file - The UNIX and Linux Forums)

mradsus

Hi All,
I created a crontab entry in a cron.txt file accidentally entered

crontab cron.txt.

Now my previous crontab -l entries are not showing up, that means i removed the scheduling of the previous jobs by running this command "crontab cron.txt"

How do I revert back to previously schedule jobs.
Please help. this is urgent.,
Thanks.

In this case, if you do not have a backup,  you only remedy is to try to extract cron commands from /var/log/messages.

For classic cron backup and restore are simple. You just pipe the crontab in backup file or pipe the backup from the crontab backup file.

To backup the current crontab settings for current user:
crontab -l >  `whoami`.crontab.`date +%y%m%d`

Or from root:

crontab -l -u USERNAME > `whoami`.crontab`date +%%y%m%d`

If we have a set of files named for example root.crontab131010root.crontab131011, root.crontab131012, you can restore content of the most recent backup of the crontab using the command:

ls -tr `whoami`.crontab* | tail -1 | crontab -u `whoami` -

The “-“ specifies that crontab will use the standard input.

With Suse and RHEL structure /etc/cron.d directory should be backed up too. Remember that crontab data can exist for each user of the system, not just root.  For example the user oracle often has crontab entries.

You can delete older files and backup new on continued basis. Here are some hints for the implementation  (How to backup and restore crontab Andres Montalban):

...I had to come up with a solution for a customer using AWS/EC2 to make their crontab resilient to server re-builds in case it goes down by autoscaler or other situations. That’s why I created a script that is called daily that backups the crontab of a specific user to a file in an EBS mounted volume maintaining a 7 days library just in case:

#!/bin/bash
find /data -maxdepth 1 -name ‘bkp-crontab-*’ -mtime 7 -exec rm {} \; 
crontab -l -u YOUR_USER_GOES_HERE > /data/bkp-crontab-`date +%Y-%m-%d`

Then to recover the last backup of crontab for the user you can put this in your server script when you are building it:

cat `ls -tr /data/bkp-crontab-* | tail -1` | crontab -u YOUR_USER_GOES_HERE -

This will load the last backup file in the users crontab.

I hope this helps you to have your crontabs backed up

You can also operate with files in /var/spoon/cron/tabs (in Linux) and /var/spoon/cron/crontabs in classic Unixes.

 


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Aug 10, 2020] How to Run and Control Background Processes on Linux

Aug 10, 2020 | www.howtogeek.com

How to Run and Control Background Processes on Linux DAVE MCKAY @thegurkha
SEPTEMBER 24, 2019, 8:00AM EDT

A shell environment on a Linux computer.
Fatmawati Achmad Zaenuri/Shutterstock.com

Use the Bash shell in Linux to manage foreground and background processes. You can use Bash's job control functions and signals to give you more flexibility in how you run commands. We show you how.

How to Speed Up a Slow PC

https://imasdk.googleapis.com/js/core/bridge3.401.2_en.html#goog_863166184 All About Processes

Whenever a program is executed in a Linux or Unix-like operating system, a process is started. "Process" is the name for the internal representation of the executing program in the computer's memory. There is a process for every active program. In fact, there is a process for nearly everything that is running on your computer. That includes the components of your graphical desktop environment (GDE) such as GNOME or KDE , and system daemons that are launched at start-up.

Why nearly everything that is running? Well, Bash built-ins such as cd , pwd , and alias do not need to have a process launched (or "spawned") when they are run. Bash executes these commands within the instance of the Bash shell that is running in your terminal window. These commands are fast precisely because they don't need to have a process launched for them to execute. (You can type help in a terminal window to see the list of Bash built-ins.)

Processes can be running in the foreground, in which case they take over your terminal until they have completed, or they can be run in the background. Processes that run in the background don't dominate the terminal window and you can continue to work in it. Or at least, they don't dominate the terminal window if they don't generate screen output.

A Messy Example

We'll start a simple ping trace running . We're going to ping the How-To Geek domain. This will execute as a foreground process.

ping www.howtogeek.com

ping www.howtogeek.com in a terminal window

We get the expected results, scrolling down the terminal window. We can't do anything else in the terminal window while ping is running. To terminate the command hit Ctrl+C .

Ctrl+C

ping trace output in a terminal window

The visible effect of the Ctrl+C is highlighted in the screenshot. ping gives a short summary and then stops.

Let's repeat that. But this time we'll hit Ctrl+Z instead of Ctrl+C . The task won't be terminated. It will become a background task. We get control of the terminal window returned to us.

ping www.howtogeek.com
Ctrl+Z

effect of Ctrl+Z on a command running in a terminal window

The visible effect of hitting Ctrl+Z is highlighted in the screenshot.

This time we are told the process is stopped. Stopped doesn't mean terminated. It's like a car at a stop sign. We haven't scrapped it and thrown it away. It's still on the road, stationary, waiting to go. The process is now a background job .

The jobs command will list the jobs that have been started in the current terminal session. And because jobs are (inevitably) processes, we can also use the ps command to see them. Let's use both commands and compare their outputs. We'll use the T option (terminal) option to only list the processes that are running in this terminal window. Note that there is no need to use a hyphen - with the T option.

jobs
ps T

jobs command in a terminal window

The jobs command tells us:

The ps command tells us:

These are common values for the STAT column:

The value in the STAT column can be followed by one of these extra indicators:

We can see that Bash has a state of Ss . The uppercase "S" tell us the Bash shell is sleeping, and it is interruptible. As soon as we need it, it will respond. The lowercase "s" tells us that the shell is a session leader.

The ping command has a state of T . This tells us that ping has been stopped by a job control signal. In this example, that was the Ctrl+Z we used to put it into the background.

The ps T command has a state of R , which stands for running. The + indicates that this process is a member of the foreground group. So the ps T command is running in the foreground.

The bg Command

The bg command is used to resume a background process. It can be used with or without a job number. If you use it without a job number the default job is brought to the foreground. The process still runs in the background. You cannot send any input to it.

If we issue the bg command, we will resume our ping command:

bg

bg in a terminal window

The ping command resumes and we see the scrolling output in the terminal window once more. The name of the command that has been restarted is displayed for you. This is highlighted in the screenshot.

resumed ping background process with output in a terminal widow

But we have a problem. The task is running in the background and won't accept input. So how do we stop it? Ctrl+C doesn't do anything. We can see it when we type it but the background task doesn't receive those keystrokes so it keeps pinging merrily away.

Background task ignoring Ctrl+C in a terminal window

In fact, we're now in a strange blended mode. We can type in the terminal window but what we type is quickly swept away by the scrolling output from the ping command. Anything we type takes effect in the foregound.

To stop our background task we need to bring it to the foreground and then stop it.

The fg Command

The fg command will bring a background task into the foreground. Just like the bg command, it can be used with or without a job number. Using it with a job number means it will operate on a specific job. If it is used without a job number the last command that was sent to the background is used.

If we type fg our ping command will be brought to the foreground. The characters we type are mixed up with the output from the ping command, but they are operated on by the shell as if they had been entered on the command line as usual. And in fact, from the Bash shell's point of view, that is exactly what has happened.

fg

fg command mixed in with the output from ping in a terminal window

And now that we have the ping command running in the foreground once more, we can use Ctrl+C to kill it.

Ctrl+C

Ctrl+C stopping the ping command in a terminal window

We Need to Send the Right Signals

That wasn't exactly pretty. Evidently running a process in the background works best when the process doesn't produce output and doesn't require input.

But, messy or not, our example did accomplish:

When you use Ctrl+C and Ctrl+Z , you are sending signals to the process. These are shorthand ways of using the kill command. There are 64 different signals that kill can send. Use kill -l at the command line to list them. kill isn't the only source of these signals. Some of them are raised automatically by other processes within the system

Here are some of the commonly used ones.

We must use the kill command to issue signals that do not have key combinations assigned to them.

Further Job Control

A process moved into the background by using Ctrl+Z is placed in the stopped state. We have to use the bg command to start it running again. To launch a program as a running background process is simple. Append an ampersand & to the end of the command line.

Although it is best that background processes do not write to the terminal window, we're going to use examples that do. We need to have something in the screenshots that we can refer to. This command will start an endless loop as a background process:

while true; do echo "How-To Geek Loop Process"; sleep 3; done &

while true; do echo "How-To Geek Loop Process"; sleep 3; done & in a terminal window

We are told the job number and process ID id of the process. Our job number is 1, and the process id is 1979. We can use these identifiers to control the process.

The output from our endless loop starts to appear in the terminal window. As before, we can use the command line but any commands we issue are interspersed with the output from the loop process.

ls

output of the background loop process interspersed with output from other commands

To stop our process we can use jobs to remind ourselves what the job number is, and then use kill .

jobs reports that our process is job number 1. To use that number with kill we must precede it with a percent sign % .

jobs
kill %1

jobs and kill %1 in a terminal window

kill sends the SIGTERM signal, signal number 15, to the process and it is terminated. When the Enter key is next pressed, a status of the job is shown. It lists the process as "terminated." If the process does not respond to the kill command you can take it up a notch. Use kill with SIGKILL , signal number 9. Just put the number 9 between the kill command the job number.

kill 9 %1
Things We've Covered

RELATED: How to Kill Processes From the Linux Terminal

[Jul 29, 2020] Linux Commands- jobs, bg, and fg by Tyler Carrigan

Jul 23, 2020 | www.redhat.com
Image

Photo by Andrea Piacquadio from Pexels

More Linux resources

In this quick tutorial, I want to look at the jobs command and a few of the ways that we can manipulate the jobs running on our systems. In short, controlling jobs lets you suspend and resume processes started in your Linux shell.

Jobs

The jobs command will list all jobs on the system; active, stopped, or otherwise. Before I explore the command and output, I'll create a job on my system.

I will use the sleep job as it won't change my system in any meaningful way.

[tcarrigan@rhel ~]$ sleep 500
^Z
[1]+  Stopped                 sleep 500

First, I issued the sleep command, and then I received the Job number [1]. I then immediately stopped the job by using Ctl+Z . Next, I run the jobs command to view the newly created job:

[tcarrigan@rhel ~]$ jobs
[1]+  Stopped                 sleep 500

You can see that I have a single stopped job identified by the job number [1] .

Other options to know for this command include:

Background

Next, I'll resume the sleep job in the background. To do this, I use the bg command. Now, the bg command has a pretty simple syntax, as seen here:

bg [JOB_SPEC]

Where JOB_SPEC can be any of the following:

NOTE : bg and fg operate on the current job if no JOB_SPEC is provided.

I can move this job to the background by using the job number [1] .

[tcarrigan@rhel ~]$ bg %1
[1]+ sleep 500 &

You can see now that I have a single running job in the background.

[tcarrigan@rhel ~]$ jobs
[1]+  Running                 sleep 500 &
Foreground

Now, let's look at how to move a background job into the foreground. To do this, I use the fg command. The command syntax is the same for the foreground command as with the background command.

fg [JOB_SPEC]

Refer to the above bullets for details on JOB_SPEC.

I have started a new sleep in the background:

[tcarrigan@rhel ~]$ sleep 500 &
[2] 5599

Now, I'll move it to the foreground by using the following command:

[tcarrigan@rhel ~]$ fg %2
sleep 500

The fg command has now brought my system back into a sleep state.

The end

While I realize that the jobs presented here were trivial, these concepts can be applied to more than just the sleep command. If you run into a situation that requires it, you now have the knowledge to move running or stopped jobs from the foreground to background and back again.

[Nov 08, 2019] How to use cron in Linux by David Both

Nov 06, 2017 | opensource.com
No time for commands? Scheduling tasks with cron means programs can run but you don't have to stay up late. 9 comments Image by : Internet Archive Book Images. Modified by Opensource.com. CC BY-SA 4.0 x Subscribe now

Get the highlights in your inbox every week.

https://opensource.com/eloqua-embedded-email-capture-block.html?offer_id=70160000000QzXNAA0

Instead, I use two service utilities that allow me to run commands, programs, and tasks at predetermined times. The cron and at services enable sysadmins to schedule tasks to run at a specific time in the future. The at service specifies a one-time task that runs at a certain time. The cron service can schedule tasks on a repetitive basis, such as daily, weekly, or monthly.

In this article, I'll introduce the cron service and how to use it.

Common (and uncommon) cron uses

I use the cron service to schedule obvious things, such as regular backups that occur daily at 2 a.m. I also use it for less obvious things.

The crond daemon is the background service that enables cron functionality.

The cron service checks for files in the /var/spool/cron and /etc/cron.d directories and the /etc/anacrontab file. The contents of these files define cron jobs that are to be run at various intervals. The individual user cron files are located in /var/spool/cron , and system services and applications generally add cron job files in the /etc/cron.d directory. The /etc/anacrontab is a special case that will be covered later in this article.

Using crontab

The cron utility runs based on commands specified in a cron table ( crontab ). Each user, including root, can have a cron file. These files don't exist by default, but can be created in the /var/spool/cron directory using the crontab -e command that's also used to edit a cron file (see the script below). I strongly recommend that you not use a standard editor (such as Vi, Vim, Emacs, Nano, or any of the many other editors that are available). Using the crontab command not only allows you to edit the command, it also restarts the crond daemon when you save and exit the editor. The crontab command uses Vi as its underlying editor, because Vi is always present (on even the most basic of installations).

New cron files are empty, so commands must be added from scratch. I added the job definition example below to my own cron files, just as a quick reference, so I know what the various parts of a command mean. Feel free to copy it for your own use.

# crontab -e
SHELL = / bin / bash
MAILTO =root @ example.com
PATH = / bin: / sbin: / usr / bin: / usr / sbin: / usr / local / bin: / usr / local / sbin

# For details see man 4 crontabs

# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed

# backup using the rsbu program to the internal 4TB HDD and then 4TB external
01 01 * * * / usr / local / bin / rsbu -vbd1 ; / usr / local / bin / rsbu -vbd2

# Set the hardware clock to keep it in sync with the more accurate system clock
03 05 * * * / sbin / hwclock --systohc

# Perform monthly updates on the first of the month
# 25 04 1 * * /usr/bin/dnf -y update

The crontab command is used to view or edit the cron files.

The first three lines in the code above set up a default environment. The environment must be set to whatever is necessary for a given user because cron does not provide an environment of any kind. The SHELL variable specifies the shell to use when commands are executed. This example specifies the Bash shell. The MAILTO variable sets the email address where cron job results will be sent. These emails can provide the status of the cron job (backups, updates, etc.) and consist of the output you would see if you ran the program manually from the command line. The third line sets up the PATH for the environment. Even though the path is set here, I always prepend the fully qualified path to each executable.

There are several comment lines in the example above that detail the syntax required to define a cron job. I'll break those commands down, then add a few more to show you some more advanced capabilities of crontab files.

01 01 * * * /usr/local/bin/rsbu -vbd1 ; /usr/local/bin/rsbu -vbd2

This line in my /etc/crontab runs a script that performs backups for my systems.

This line runs my self-written Bash shell script, rsbu , that backs up all my systems. This job kicks off at 1:01 a.m. (01 01) every day. The asterisks (*) in positions three, four, and five of the time specification are like file globs, or wildcards, for other time divisions; they specify "every day of the month," "every month," and "every day of the week." This line runs my backups twice; one backs up to an internal dedicated backup hard drive, and the other backs up to an external USB drive that I can take to the safe deposit box.

The following line sets the hardware clock on the computer using the system clock as the source of an accurate time. This line is set to run at 5:03 a.m. (03 05) every day.

03 05 * * * /sbin/hwclock --systohc

This line sets the hardware clock using the system time as the source.

I was using the third and final cron job (commented out) to perform a dnf or yum update at 04:25 a.m. on the first day of each month, but I commented it out so it no longer runs.

# 25 04 1 * * /usr/bin/dnf -y update

This line used to perform a monthly update, but I've commented it out.

Other scheduling tricks

Now let's do some things that are a little more interesting than these basics. Suppose you want to run a particular job every Thursday at 3 p.m.:

00 15 * * Thu /usr/local/bin/mycronjob.sh

This line runs mycronjob.sh every Thursday at 3 p.m.

Or, maybe you need to run quarterly reports after the end of each quarter. The cron service has no option for "The last day of the month," so instead you can use the first day of the following month, as shown below. (This assumes that the data needed for the reports will be ready when the job is set to run.)

02 03 1 1,4,7,10 * /usr/local/bin/reports.sh

This cron job runs quarterly reports on the first day of the month after a quarter ends.

The following shows a job that runs one minute past every hour between 9:01 a.m. and 5:01 p.m.

01 09-17 * * * /usr/local/bin/hourlyreminder.sh

Sometimes you want to run jobs at regular times during normal business hours.

I have encountered situations where I need to run a job every two, three, or four hours. That can be accomplished by dividing the hours by the desired interval, such as */3 for every three hours, or 6-18/3 to run every three hours between 6 a.m. and 6 p.m. Other intervals can be divided similarly; for example, the expression */15 in the minutes position means "run the job every 15 minutes."

*/5 08-18/2 * * * /usr/local/bin/mycronjob.sh

This cron job runs every five minutes during every hour between 8 a.m. and 5:58 p.m.

One thing to note: The division expressions must result in a remainder of zero for the job to run. That's why, in this example, the job is set to run every five minutes (08:05, 08:10, 08:15, etc.) during even-numbered hours from 8 a.m. to 6 p.m., but not during any odd-numbered hours. For example, the job will not run at all from 9 p.m. to 9:59 a.m.

I am sure you can come up with many other possibilities based on these examples.

Limiting cron access

More Linux resources

Regular users with cron access could make mistakes that, for example, might cause system resources (such as memory and CPU time) to be swamped. To prevent possible misuse, the sysadmin can limit user access by creating a /etc/cron.allow file that contains a list of all users with permission to create cron jobs. The root user cannot be prevented from using cron.

By preventing non-root users from creating their own cron jobs, it may be necessary for root to add their cron jobs to the root crontab. "But wait!" you say. "Doesn't that run those jobs as root?" Not necessarily. In the first example in this article, the username field shown in the comments can be used to specify the user ID a job is to have when it runs. This prevents the specified non-root user's jobs from running as root. The following example shows a job definition that runs a job as the user "student":

04 07 * * * student /usr/local/bin/mycronjob.sh

If no user is specified, the job is run as the user that owns the crontab file, root in this case.

cron.d

The directory /etc/cron.d is where some applications, such as SpamAssassin and sysstat , install cron files. Because there is no spamassassin or sysstat user, these programs need a place to locate cron files, so they are placed in /etc/cron.d .

The /etc/cron.d/sysstat file below contains cron jobs that relate to system activity reporting (SAR). These cron files have the same format as a user cron file.

# Run system activity accounting tool every 10 minutes
*/ 10 * * * * root / usr / lib64 / sa / sa1 1 1
# Generate a daily summary of process accounting at 23:53
53 23 * * * root / usr / lib64 / sa / sa2 -A

The sysstat package installs the /etc/cron.d/sysstat cron file to run programs for SAR.

The sysstat cron file has two lines that perform tasks. The first line runs the sa1 program every 10 minutes to collect data stored in special binary files in the /var/log/sa directory. Then, every night at 23:53, the sa2 program runs to create a daily summary.

Scheduling tips

Some of the times I set in the crontab files seem rather random -- and to some extent they are. Trying to schedule cron jobs can be challenging, especially as the number of jobs increases. I usually have only a few tasks to schedule on each of my computers, which is simpler than in some of the production and lab environments where I have worked.

One system I administered had around a dozen cron jobs that ran every night and an additional three or four that ran on weekends or the first of the month. That was a challenge, because if too many jobs ran at the same time -- especially the backups and compiles -- the system would run out of RAM and nearly fill the swap file, which resulted in system thrashing while performance tanked, so nothing got done. We added more memory and improved how we scheduled tasks. We also removed a task that was very poorly written and used large amounts of memory.

The crond service assumes that the host computer runs all the time. That means that if the computer is turned off during a period when cron jobs were scheduled to run, they will not run until the next time they are scheduled. This might cause problems if they are critical cron jobs. Fortunately, there is another option for running jobs at regular intervals: anacron .

anacron

The anacron program performs the same function as crond, but it adds the ability to run jobs that were skipped, such as if the computer was off or otherwise unable to run the job for one or more cycles. This is very useful for laptops and other computers that are turned off or put into sleep mode.

As soon as the computer is turned on and booted, anacron checks to see whether configured jobs missed their last scheduled run. If they have, those jobs run immediately, but only once (no matter how many cycles have been missed). For example, if a weekly job was not run for three weeks because the system was shut down while you were on vacation, it would be run soon after you turn the computer on, but only once, not three times.

The anacron program provides some easy options for running regularly scheduled tasks. Just install your scripts in the /etc/cron.[hourly|daily|weekly|monthly] directories, depending how frequently they need to be run.

How does this work? The sequence is simpler than it first appears.

  1. The crond service runs the cron job specified in /etc/cron.d/0hourly .
# Run the hourly jobs
SHELL = / bin / bash
PATH = / sbin: / bin: / usr / sbin: / usr / bin
MAILTO =root
01 * * * * root run-parts / etc / cron.hourly

The contents of /etc/cron.d/0hourly cause the shell scripts located in /etc/cron.hourly to run.

  1. The cron job specified in /etc/cron.d/0hourly runs the run-parts program once per hour.
  2. The run-parts program runs all the scripts located in the /etc/cron.hourly directory.
  3. The /etc/cron.hourly directory contains the 0anacron script, which runs the anacron program using the /etdc/anacrontab configuration file shown here.
# /etc/anacrontab: configuration file for anacron

# See anacron(8) and anacrontab(5) for details.

SHELL = / bin / sh
PATH = / sbin: / bin: / usr / sbin: / usr / bin
MAILTO =root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY = 45
# the jobs will be started during the following hours only
START_HOURS_RANGE = 3 - 22

#period in days delay in minutes job-identifier command
1 5 cron.daily nice run-parts / etc / cron.daily
7 25 cron.weekly nice run-parts / etc / cron.weekly
@ monthly 45 cron.monthly nice run-parts / etc / cron.monthly

The contents of /etc/anacrontab file runs the executable files in the cron.[daily|weekly|monthly] directories at the appropriate times.

  1. The anacron program runs the programs located in /etc/cron.daily once per day; it runs the jobs located in /etc/cron.weekly once per week, and the jobs in cron.monthly once per month. Note the specified delay times in each line that help prevent these jobs from overlapping themselves and other cron jobs.

Instead of placing complete Bash programs in the cron.X directories, I install them in the /usr/local/bin directory, which allows me to run them easily from the command line. Then I add a symlink in the appropriate cron directory, such as /etc/cron.daily .

The anacron program is not designed to run programs at specific times. Rather, it is intended to run programs at intervals that begin at the specified times, such as 3 a.m. (see the START_HOURS_RANGE line in the script just above) of each day, on Sunday (to begin the week), and on the first day of the month. If any one or more cycles are missed, anacron will run the missed jobs once, as soon as possible.

More on setting limits

I use most of these methods for scheduling tasks to run on my computers. All those tasks are ones that need to run with root privileges. It's rare in my experience that regular users really need a cron job. One case was a developer user who needed a cron job to kick off a daily compile in a development lab.

It is important to restrict access to cron functions by non-root users. However, there are circumstances when a user needs to set a task to run at pre-specified times, and cron can allow them to do that. Many users do not understand how to properly configure these tasks using cron and they make mistakes. Those mistakes may be harmless, but, more often than not, they can cause problems. By setting functional policies that cause users to interact with the sysadmin, individual cron jobs are much less likely to interfere with other users and other system functions.

It is possible to set limits on the total resources that can be allocated to individual users or groups, but that is an article for another time.

For more information, the man pages for cron , crontab , anacron , anacrontab , and run-parts all have excellent information and descriptions of how the cron system works.


Ben Cotton on 06 Nov 2017 Permalink

One problem I used to have in an old job was cron jobs that would hang for some reason. This old sysadvent post had some good suggestions for how to deal with that: http://sysadvent.blogspot.com/2009/12/cron-practices.html

Jesper Larsen on 06 Nov 2017 Permalink

Cron is definitely a good tool. But if you need to do more advanced scheduling then Apache Airflow is great for this.

Airflow has a number of advantages over Cron. The most important are: Dependencies (let tasks run after other tasks), nice web based overview, automatic failure recovery and a centralized scheduler. The disadvantages are that you will need to setup the scheduler and some other centralized components on one server and a worker on each machine you want to run stuff on.

You definitely want to use Cron for some stuff. But if you find that Cron is too limited for your use case I would recommend looking into Airflow.

Leslle Satenstein on 13 Nov 2017 Permalink

Hi David,
you have a well done article. Much appreciated. I make use of the @reboot crontab entry. With crontab and root. I run the following.

@reboot /bin/dofstrim.sh

I wanted to run fstrim for my SSD drive once and only once per week.
dofstrim.sh is a script that runs the "fstrim" program once per week, irrespective of the number of times the system is rebooted. I happen to have several Linux systems sharing one computer, and each system has a root crontab with that entry. Since I may hop from Linux to Linux in the day or several times per week, my dofstrim.sh only runs fstrim once per week, irrespective which Linux system I boot. I make use of a common partition to all Linux systems, a partition mounted as "/scratch" and the wonderful Linux command line "date" program.

The dofstrim.sh listing follows below.

#!/bin/bash
# run fstrim either once/week or once/day not once for every reboot
#
# Use the date function to extract today's day number or week number
# the day number range is 1..366, weekno is 1 to 53
#WEEKLY=0 #once per day
WEEKLY=1 #once per week
lockdir='/scratch/lock/'

if [[ WEEKLY -eq 1 ]]; then
dayno="$lockdir/dofstrim.weekno"
today=$(date +%V)
else
dayno=$lockdir/dofstrim.dayno
today=$(date +%j)
fi

prevval="000"

if [ -f "$dayno" ]
then
prevval=$(cat ${dayno} )
if [ x$prevval = x ];then
prevval="000"
fi
else
mkdir -p $lockdir
fi

if [ ${prevval} -ne ${today} ]
then
/sbin/fstrim -a
echo $today > $dayno
fi

I had thought to use anacron, but then fstrim would be run frequently as each linux's anacron would have a similar entry.
The "date" program produces a day number or a week number, depending upon the +%V or +%j

Leslle Satenstein on 13 Nov 2017 Permalink

Running a report on the last day of the month is easy if you use the date program. Use the date function from Linux as shown

*/9 15 28-31 * * [ `date -d +'1 day' +\%d` -eq 1 ] && echo "Tomorrow is the first of month Today(now) is `date`" >> /root/message

Once per day from the 28th to the 31st, the date function is executed.
If the result of date +1day is the first of the month, today must be the last day of the month.

sgtrock on 14 Nov 2017 Permalink

Why not use crontab to launch something like Ansible playbooks instead of simple bash scripts? A lot easier to troubleshoot and manage these days. :-)

[Jun 23, 2018] Queuing tasks for batch execution with Task Spooler by Ben Martin

Aug 12, 2008 | www.linux.com

The Task Spooler project allows you to queue up tasks from the shell for batch execution. Task Spooler is simple to use and requires no configuration. You can view and edit queued commands, and you can view the output of queued commands at any time.

Task Spooler has some similarities with other delayed and batch execution projects, such as " at ." While both Task Spooler and at handle multiple queues and allow the execution of commands at a later point, the at project handles output from commands by emailing the results to the user who queued the command, while Task Spooler allows you to get at the results from the command line instead. Another major difference is that Task Spooler is not aimed at executing commands at a specific time, but rather at simply adding to and executing commands from queues.

The main repositories for Fedora, openSUSE, and Ubuntu do not contain packages for Task Spooler. There are packages for some versions of Debian, Ubuntu, and openSUSE 10.x available along with the source code on the project's homepage. In this article I'll use a 64-bit Fedora 9 machine and install version 0.6 of Task Spooler from source. Task Spooler does not use autotools to build, so to install it, simply run make; sudo make install . This will install the main Task Spooler command ts and its manual page into /usr/local.

A simple interaction with Task Spooler is shown below. First I add a new job to the queue and check the status. As the command is a very simple one, it is likely to have been executed immediately. Executing ts by itself with no arguments shows the executing queue, including tasks that have completed. I then use ts -c to get at the stdout of the executed command. The -c option uses cat to display the output file for a task. Using ts -i shows you information about the job. To clear finished jobs from the queue, use the ts -C command, not shown in the example.

$ ts echo "hello world"
6

$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/1]
6 finished /tmp/ts-out.QoKfo9 0 0.00/0.00/0.00 echo hello world

$ ts -c 6
hello world

$ ts -i 6
Command: echo hello world
Enqueue time: Tue Jul 22 14:42:22 2008
Start time: Tue Jul 22 14:42:22 2008
End time: Tue Jul 22 14:42:22 2008
Time run: 0.003336s

The -t option operates like tail -f , showing you the last few lines of output and continuing to show you any new output from the task. If you would like to be notified when a task has completed, you can use the -m option to have the results mailed to you, or you can queue another command to be executed that just performs the notification. For example, I might add a tar command and want to know when it has completed. The below commands will create a tarball and use libnotify commands to create an inobtrusive popup window on my desktop when the tarball creation is complete. The popup will be dismissed automatically after a timeout.

$ ts tar czvf /tmp/mytarball.tar.gz liberror-2.1.80011
11
$ ts notify-send "tarball creation" "the long running tar creation process is complete."
12
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/1]
11 finished /tmp/ts-out.O6epsS 0 4.64/4.31/0.29 tar czvf /tmp/mytarball.tar.gz liberror-2.1.80011
12 finished /tmp/ts-out.4KbPSE 0 0.05/0.00/0.02 notify-send tarball creation the long... is complete.

Notice in the output above, toward the far right of the header information, the run=0/1 line. This tells you that Task Spooler is executing nothing, and can possibly execute one task. Task spooler allows you to execute multiple tasks at once from your task queue to take advantage of multicore CPUs. The -S option allows you to set how many tasks can be executed in parallel from the queue, as shown below.

$ ts -S 2
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/2]
6 finished /tmp/ts-out.QoKfo9 0 0.00/0.00/0.00 echo hello world

If you have two tasks that you want to execute with Task Spooler but one depends on the other having already been executed (and perhaps that the previous job has succeeded too) you can handle this by having one task wait for the other to complete before executing. This becomes more important on a quad core machine when you might have told Task Spooler that it can execute three tasks in parallel. The commands shown below create an explicit dependency, making sure that the second command is executed only if the first has completed successfully, even when the queue allows multiple tasks to be executed. The first command is queued normally using ts . I use a subshell to execute the commands by having ts explicitly start a new bash shell. The second command uses the -d option, which tells ts to execute the command only after the successful completion of the last command that was appended to the queue. When I first inspect the queue I can see that the first command (28) is executing. The second command is queued but has not been added to the list of executing tasks because Task Spooler is aware that it cannot execute until task 28 is complete. The second time I view the queue, both tasks have completed.

$ ts bash -c "sleep 10; echo hi"
28
$ ts -d echo there
29
$ ts
ID State Output E-Level Times(r/u/s) Command [run=1/2]
28 running /tmp/ts-out.hKqDva bash -c sleep 10; echo hi
29 queued (file) && echo there
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/2]
28 finished /tmp/ts-out.hKqDva 0 10.01/0.00/0.01 bash -c sleep 10; echo hi
29 finished /tmp/ts-out.VDtVp7 0 0.00/0.00/0.00 && echo there
$ cat /tmp/ts-out.hKqDva
hi
$ cat /tmp/ts-out.VDtVp7
there

You can also explicitly set dependencies on other tasks as shown below. Because the ts command prints the ID of a new task to the console, the first command puts that ID into a shell variable for use in the second command. The second command passes the task ID of the first task to ts, telling it to wait for the task with that ID to complete before returning. Because this is joined with the command we wish to execute with the && operation, the second command will execute only if the first one has finished and succeeded.

The first time we view the queue you can see that both tasks are running. The first task will be in the sleep command that we used explicitly to slow down its execution. The second command will be executing ts , which will be waiting for the first task to complete. One downside of tracking dependencies this way is that the second command is added to the running queue even though it cannot do anything until the first task is complete.

$ FIRST_TASKID=`ts bash -c "sleep 10; echo hi"`
$ ts sh -c "ts -w $FIRST_TASKID && echo there"
25
$ ts
ID State Output E-Level Times(r/u/s) Command [run=2/2]
24 running /tmp/ts-out.La9Gmz bash -c sleep 10; echo hi
25 running /tmp/ts-out.Zr2n5u sh -c ts -w 24 && echo there
$ ts
ID State Output E-Level Times(r/u/s) Command [run=0/2]
24 finished /tmp/ts-out.La9Gmz 0 10.01/0.00/0.00 bash -c sleep 10; echo hi
25 finished /tmp/ts-out.Zr2n5u 0 9.47/0.00/0.01 sh -c ts -w 24 && echo there
$ ts -c 24
hi
$ ts -c 25
there Wrap-up

Task Spooler allows you to convert a shell command to a queued command by simply prepending ts to the command line. One major advantage of using ts over something like the at command is that you can effectively run tail -f on the output of a running task and also get at the output of completed tasks from the command line. The utility's ability to execute multiple tasks in parallel is very handy if you are running on a multicore CPU. Because you can explicitly wait for a task, you can set up very complex interactions where you might have several tasks running at once and have jobs that depend on multiple other tasks to complete successfully before they can execute.

Because you can make explicitly dependant tasks take up slots in the actively running task queue, you can effectively delay the execution of the queue until a time of your choosing. For example, if you queue up a task that waits for a specific time before returning successfully and have a small group of other tasks that are dependent on this first task to complete, then no tasks in the queue will execute until the first task completes.

Category:

Click Here!

[Jun 23, 2018] at, batch, atq, and atrm examples

Jun 23, 2018 | www.computerhope.com
at -m 01:35 < my-at-jobs.txt

Run the commands listed in the ' my-at-jobs.txt ' file at 1:35 AM. All output from the job will be mailed to the user running the task. When this command has been successfully entered you should receive a prompt similar to the example below:

commands will be executed using /bin/sh
job 1 at Wed Dec 24 00:22:00 2014
at -l

This command will list each of the scheduled jobs in a format like the following:

1          Wed Dec 24 00:22:00 2003

...this is the same as running the command atq .

at -r 1

Deletes job 1 . This command is the same as running the command atrm 1 .

atrm 23

Deletes job 23. This command is the same as running the command at -r 23 .

[Jun 23, 2018] Bash script processing limited number of commands in parallel

Jun 23, 2018 | stackoverflow.com

AL-Kateb ,Oct 23, 2013 at 13:33

I have a bash script that looks like this:
#!/bin/bash
wget LINK1 >/dev/null 2>&1
wget LINK2 >/dev/null 2>&1
wget LINK3 >/dev/null 2>&1
wget LINK4 >/dev/null 2>&1
# ..
# ..
wget LINK4000 >/dev/null 2>&1

But processing each line until the command is finished then moving to the next one is very time consuming, I want to process for instance 20 lines at once then when they're finished another 20 lines are processed.

I thought of wget LINK1 >/dev/null 2>&1 & to send the command to the background and carry on, but there are 4000 lines here this means I will have performance issues, not to mention being limited in how many processes I should start at the same time so this is not a good idea.

One solution that I'm thinking of right now is checking whether one of the commands is still running or not, for instance after 20 lines I can add this loop:

while [  $(ps -ef | grep KEYWORD | grep -v grep | wc -l) -gt 0 ]; do
sleep 1
done

Of course in this case I will need to append & to the end of the line! But I'm feeling this is not the right way to do it.

So how do I actually group each 20 lines together and wait for them to finish before going to the next 20 lines, this script is dynamically generated so I can do whatever math I want on it while it's being generated, but it DOES NOT have to use wget, it was just an example so any solution that is wget specific is not gonna do me any good.

kojiro ,Oct 23, 2013 at 13:46

wait is the right answer here, but your while [ $(ps would be much better written while pkill -0 $KEYWORD – using proctools that is, for legitimate reasons to check if a process with a specific name is still running. – kojiro Oct 23 '13 at 13:46

VasyaNovikov ,Jan 11 at 19:01

I think this question should be re-opened. The "possible duplicate" QA is all about running a finite number of programs in parallel. Like 2-3 commands. This question, however, is focused on running commands in e.g. a loop. (see "but there are 4000 lines"). – VasyaNovikov Jan 11 at 19:01

robinCTS ,Jan 11 at 23:08

@VasyaNovikov Have you read all the answers to both this question and the duplicate? Every single answer to this question here, can also be found in the answers to the duplicate question. That is precisely the definition of a duplicate question. It makes absolutely no difference whether or not you are running the commands in a loop. – robinCTS Jan 11 at 23:08

VasyaNovikov ,Jan 12 at 4:09

@robinCTS there are intersections, but questions themselves are different. Also, 6 of the most popular answers on the linked QA deal with 2 processes only. – VasyaNovikov Jan 12 at 4:09

Dan Nissenbaum ,Apr 20 at 15:35

I recommend reopening this question because its answer is clearer, cleaner, better, and much more highly upvoted than the answer at the linked question, though it is three years more recent. – Dan Nissenbaum Apr 20 at 15:35

devnull ,Oct 23, 2013 at 13:35

Use the wait built-in:
process1 &
process2 &
process3 &
process4 &
wait
process5 &
process6 &
process7 &
process8 &
wait

For the above example, 4 processes process1 .. process4 would be started in the background, and the shell would wait until those are completed before starting the next set ..

From the manual :

wait [jobspec or pid ...]

Wait until the child process specified by each process ID pid or job specification jobspec exits and return the exit status of the last command waited for. If a job spec is given, all processes in the job are waited for. If no arguments are given, all currently active child processes are waited for, and the return status is zero. If neither jobspec nor pid specifies an active child process of the shell, the return status is 127.

kojiro ,Oct 23, 2013 at 13:48

So basically i=0; waitevery=4; for link in "${links[@]}"; do wget "$link" & (( i++%waitevery==0 )) && wait; done >/dev/null 2>&1kojiro Oct 23 '13 at 13:48

rsaw ,Jul 18, 2014 at 17:26

Unless you're sure that each process will finish at the exact same time, this is a bad idea. You need to start up new jobs to keep the current total jobs at a certain cap .... parallel is the answer. – rsaw Jul 18 '14 at 17:26

DomainsFeatured ,Sep 13, 2016 at 22:55

Is there a way to do this in a loop? – DomainsFeatured Sep 13 '16 at 22:55

Bobby ,Apr 27, 2017 at 7:55

I've tried this but it seems that variable assignments done in one block are not available in the next block. Is this because they are separate processes? Is there a way to communicate the variables back to the main process? – Bobby Apr 27 '17 at 7:55

choroba ,Oct 23, 2013 at 13:38

See parallel . Its syntax is similar to xargs , but it runs the commands in parallel.

chepner ,Oct 23, 2013 at 14:35

This is better than using wait , since it takes care of starting new jobs as old ones complete, instead of waiting for an entire batch to finish before starting the next. – chepner Oct 23 '13 at 14:35

Mr. Llama ,Aug 13, 2015 at 19:30

For example, if you have the list of links in a file, you can do cat list_of_links.txt | parallel -j 4 wget {} which will keep four wget s running at a time. – Mr. Llama Aug 13 '15 at 19:30

0x004D44 ,Nov 2, 2015 at 21:42

There is a new kid in town called pexec which is a replacement for parallel . – 0x004D44 Nov 2 '15 at 21:42

mat ,Mar 1, 2016 at 21:04

Not to be picky, but xargs can also parallelize commands. – mat Mar 1 '16 at 21:04

Vader B ,Jun 27, 2016 at 6:41

In fact, xargs can run commands in parallel for you. There is a special -P max_procs command-line option for that. See man xargs .

> ,

You can run 20 processes and use the command:
wait

Your script will wait and continue when all your background jobs are finished.

[Jun 23, 2018] parallelism - correct xargs parallel usage

Jun 23, 2018 | unix.stackexchange.com

Yan Zhu ,Apr 19, 2015 at 6:59

I am using xargs to call a python script to process about 30 million small files. I hope to use xargs to parallelize the process. The command I am using is:
find ./data -name "*.json" -print0 |
  xargs -0 -I{} -P 40 python Convert.py {} > log.txt

Basically, Convert.py will read in a small json file (4kb), do some processing and write to another 4kb file. I am running on a server with 40 CPU cores. And no other CPU-intense process is running on this server.

By monitoring htop (btw, is there any other good way to monitor the CPU performance?), I find that -P 40 is not as fast as expected. Sometimes all cores will freeze and decrease almost to zero for 3-4 seconds, then will recover to 60-70%. Then I try to decrease the number of parallel processes to -P 20-30 , but it's still not very fast. The ideal behavior should be linear speed-up. Any suggestions for the parallel usage of xargs ?

Ole Tange ,Apr 19, 2015 at 8:45

You are most likely hit by I/O: The system cannot read the files fast enough. Try starting more than 40: This way it will be fine if some of the processes have to wait for I/O. – Ole Tange Apr 19 '15 at 8:45

Fox ,Apr 19, 2015 at 10:30

What kind of processing does the script do? Any database/network/io involved? How long does it run? – Fox Apr 19 '15 at 10:30

PSkocik ,Apr 19, 2015 at 11:41

I second @OleTange. That is the expected behavior if you run as many processes as you have cores and your tasks are IO bound. First the cores will wait on IO for their task (sleep), then they will process, and then repeat. If you add more processes, then the additional processes that currently aren't running on a physical core will have kicked off parallel IO operations, which will, when finished, eliminate or at least reduce the sleep periods on your cores. – PSkocik Apr 19 '15 at 11:41

Bichoy ,Apr 20, 2015 at 3:32

1- Do you have hyperthreading enabled? 2- in what you have up there, log.txt is actually overwritten with each call to convert.py ... not sure if this is the intended behavior or not. – Bichoy Apr 20 '15 at 3:32

Ole Tange ,May 11, 2015 at 18:38

xargs -P and > is opening up for race conditions because of the half-line problem gnu.org/software/parallel/ Using GNU Parallel instead will not have that problem. – Ole Tange May 11 '15 at 18:38

James Scriven ,Apr 24, 2015 at 18:00

I'd be willing to bet that your problem is python . You didn't say what kind of processing is being done on each file, but assuming you are just doing in-memory processing of the data, the running time will be dominated by starting up 30 million python virtual machines (interpreters).

If you can restructure your python program to take a list of files, instead of just one, you will get a huge improvement in performance. You can then still use xargs to further improve performance. For example, 40 processes, each processing 1000 files:

find ./data -name "*.json" -print0 |
  xargs -0 -L1000 -P 40 python Convert.py

This isn't to say that python is a bad/slow language; it's just not optimized for startup time. You'll see this with any virtual machine-based or interpreted language. Java, for example, would be even worse. If your program was written in C, there would still be a cost of starting a separate operating system process to handle each file, but it would be much less.

From there you can fiddle with -P to see if you can squeeze out a bit more speed, perhaps by increasing the number of processes to take advantage of idle processors while data is being read/written.

Stephen ,Apr 24, 2015 at 13:03

So firstly, consider the constraints:

What is the constraint on each job? If it's I/O you can probably get away with multiple jobs per CPU core up till you hit the limit of I/O, but if it's CPU intensive, its going to be worse than pointless running more jobs concurrently than you have CPU cores.

My understanding of these things is that GNU Parallel would give you better control over the queue of jobs etc.

See GNU parallel vs & (I mean background) vs xargs -P for a more detailed explanation of how the two differ.

,

As others said, check whether you're I/O-bound. Also, xargs' man page suggests using -n with -P , you don't mention the number of Convert.py processes you see running in parallel.

As a suggestion, if you're I/O-bound, you might try using an SSD block device, or try doing the processing in a tmpfs (of course, in this case you should check for enough memory, avoiding swap due to tmpfs pressure (I think), and the overhead of copying the data to it in the first place).

[Jun 23, 2018] Linux/Bash, how to schedule commands in a FIFO queue?

Jun 23, 2018 | superuser.com

Andrei ,Apr 10, 2013 at 14:26

I want the ability to schedule commands to be run in a FIFO queue. I DON'T want them to be run at a specified time in the future as would be the case with the "at" command. I want them to start running now, but not simultaneously. The next scheduled command in the queue should be run only after the first command finishes executing. Alternatively, it would be nice if I could specify a maximum number of commands from the queue that could be run simultaneously; for example if the maximum number of simultaneous commands is 2, then only at most 2 commands scheduled in the queue would be taken from the queue in a FIFO manner to be executed, the next command in the remaining queue being started only when one of the currently 2 running commands finishes.

I've heard task-spooler could do something like this, but this package doesn't appear to be well supported/tested and is not in the Ubuntu standard repositories (Ubuntu being what I'm using). If that's the best alternative then let me know and I'll use task-spooler, otherwise, I'm interested to find out what's the best, easiest, most tested, bug-free, canonical way to do such a thing with bash.

UPDATE:

Simple solutions like ; or && from bash do not work. I need to schedule these commands from an external program, when an event occurs. I just don't want to have hundreds of instances of my command running simultaneously, hence the need for a queue. There's an external program that will trigger events where I can run my own commands. I want to handle ALL triggered events, I don't want to miss any event, but I also don't want my system to crash, so that's why I want a queue to handle my commands triggered from the external program.

Andrei ,Apr 11, 2013 at 11:40

Task Spooler:

http://vicerveza.homeunix.net/~viric/soft/ts/

https://launchpad.net/ubuntu/+source/task-spooler/0.7.3-1

Does the trick very well. Hopefully it will be included in Ubuntu's package repos.

Hennes ,Apr 10, 2013 at 15:00

Use ;

For example:
ls ; touch test ; ls

That will list the directory. Only after ls has run it will run touch test which will create a file named test. And only after that has finished it will run the next command. (In this case another ls which will show the old contents and the newly created file).

Similar commands are || and && .

; will always run the next command.

&& will only run the next command it the first returned success.
Example: rm -rf *.mp3 && echo "Success! All MP3s deleted!"

|| will only run the next command if the first command returned a failure (non-zero) return value. Example: rm -rf *.mp3 || echo "Error! Some files could not be deleted! Check permissions!"

If you want to run a command in the background, append an ampersand ( & ).
Example:
make bzimage &
mp3blaster sound.mp3
make mytestsoftware ; ls ; firefox ; make clean

Will run two commands int he background (in this case a kernel build which will take some time and a program to play some music). And in the foregrounds it runs another compile job and, once that is finished ls, firefox and a make clean (all sequentially)

For more details, see man bash


[Edit after comment]

in pseudo code, something like this?

Program run_queue:

While(true)
{
   Wait_for_a_signal();

   While( queue not empty )
   {
       run next command from the queue.
       remove this command from the queue.
       // If commands where added to the queue during execution then
       // the queue is not empty, keep processing them all.
   }
   // Queue is now empty, returning to wait_for_a_signal
}
// 
// Wait forever on commands and add them to a queue
// Signal run_quueu when something gets added.
//
program add_to_queue()
{
   While(true)
   {
       Wait_for_event();
       Append command to queue
       signal run_queue
   }    
}

terdon ,Apr 10, 2013 at 15:03

The easiest way would be to simply run the commands sequentially:
cmd1; cmd2; cmd3; cmdN

If you want the next command to run only if the previous command exited successfully, use && :

cmd1 && cmd2 && cmd3 && cmdN

That is the only bash native way I know of doing what you want. If you need job control (setting a number of parallel jobs etc), you could try installing a queue manager such as TORQUE but that seems like overkill if all you want to do is launch jobs sequentially.

psusi ,Apr 10, 2013 at 15:24

You are looking for at 's twin brother: batch . It uses the same daemon but instead of scheduling a specific time, the jobs are queued and will be run whenever the system load average is low.

mpy ,Apr 10, 2013 at 14:59

Apart from dedicated queuing systems (like the Sun Grid Engine ) which you can also use locally on one machine and which offer dozens of possibilities, you can use something like
 command1 && command2 && command3

which is the other extreme -- a very simple approach. The latter neither does provide multiple simultaneous processes nor gradually filling of the "queue".

Bogdan Dumitru ,May 3, 2016 at 10:12

I went on the same route searching, trying out task-spooler and so on. The best of the best is this:

GNU Parallel --semaphore --fg It also has -j for parallel jobs.

[Jun 23, 2018] Task Spooler

Notable quotes:
"... As in freshmeat.net : ..."
"... doesn't work anymore ..."
Jun 23, 2018 | vicerveza.homeunix.net

As in freshmeat.net :

task spooler is a Unix batch system where the tasks spooled run one after the other. The amount of jobs to run at once can be set at any time. Each user in each system has his own job queue. The tasks are run in the correct context (that of enqueue) from any shell/process, and its output/results can be easily watched. It is very useful when you know that your commands depend on a lot of RAM, a lot of disk use, give a lot of output, or for whatever reason it's better not to run them all at the same time, while you want to keep your resources busy for maximum benfit. Its interface allows using it easily in scripts.

For your first contact, you can read an article at linux.com , which I like as overview, guide and examples (original url) . On more advanced usage, don't neglect the TRICKS file in the package.

Features

I wrote Task Spooler because I didn't have any comfortable way of running batch jobs in my linux computer. I wanted to:

At the end, after some time using and developing ts , it can do something more:

You can look at an old (but representative) screenshot of ts-0.2.1 if you want.

Mailing list

I created a GoogleGroup for the program. You look for the archive and the join methods in the taskspooler google group page .

Alessandro Öhler once maintained a mailing list for discussing newer functionalities and interchanging use experiences. I think this doesn't work anymore , but you can look at the old archive or even try to subscribe .

How it works

The queue is maintained by a server process. This server process is started if it isn't there already. The communication goes through a unix socket usually in /tmp/ .

When the user requests a job (using a ts client), the client waits for the server message to know when it can start. When the server allows starting , this client usually forks, and runs the command with the proper environment, because the client runs run the job and not the server, like in 'at' or 'cron'. So, the ulimits, environment, pwd,. apply.

When the job finishes, the client notifies the server. At this time, the server may notify any waiting client, and stores the output and the errorlevel of the finished job.

Moreover the client can take advantage of many information from the server: when a job finishes, where does the job output go to, etc.

Download

Download the latest version (GPLv2+ licensed): ts-1.0.tar.gz - v1.0 (2016-10-19) - Changelog

Look at the version repository if you are interested in its development.

Андрей Пантюхин (Andrew Pantyukhin) maintains the BSD port .

Alessandro Öhler provided a Gentoo ebuild for 0.4 , which with simple changes I updated to the ebuild for 0.6.4 . Moreover, the Gentoo Project Sunrise already has also an ebuild ( maybe old ) for ts .

Alexander V. Inyukhin maintains unofficial debian packages for several platforms. Find the official packages in the debian package system .

Pascal Bleser packed the program for SuSE and openSuSE in RPMs for various platforms .

Gnomeye maintains the AUR package .

Eric Keller wrote a nodejs web server showing the status of the task spooler queue ( github project ).

Manual

Look at its manpage (v0.6.1). Here you also have a copy of the help for the same version:

usage: ./ts [action] [-ngfmd] [-L <lab>] [cmd...]
Env vars:
  TS_SOCKET  the path to the unix socket used by the ts command.
  TS_MAILTO  where to mail the result (on -m). Local user by default.
  TS_MAXFINISHED  maximum finished jobs in the queue.
  TS_ONFINISH  binary called on job end (passes jobid, error, outfile, command).
  TS_ENV  command called on enqueue. Its output determines the job information.
  TS_SAVELIST  filename which will store the list, if the server dies.
  TS_SLOTS   amount of jobs which can run at once, read on server start.
Actions:
  -K       kill the task spooler server
  -C       clear the list of finished jobs
  -l       show the job list (default action)
  -S [num] set the number of max simultanious jobs of the server.
  -t [id]  tail -f the output of the job. Last run if not specified.
  -c [id]  cat the output of the job. Last run if not specified.
  -p [id]  show the pid of the job. Last run if not specified.
  -o [id]  show the output file. Of last job run, if not specified.
  -i [id]  show job information. Of last job run, if not specified.
  -s [id]  show the job state. Of the last added, if not specified.
  -r [id]  remove a job. The last added, if not specified.
  -w [id]  wait for a job. The last added, if not specified.
  -u [id]  put that job first. The last added, if not specified.
  -U <id-id>  swap two jobs in the queue.
  -h       show this help
  -V       show the program version
Options adding jobs:
  -n       don't store the output of the command.
  -g       gzip the stored output (if not -n).
  -f       don't fork into background.
  -m       send the output by e-mail (uses sendmail).
  -d       the job will be run only if the job before ends well
  -L <lab> name this task with a label, to be distinguished on listing.
Thanks

[Nov 01, 2017] Cron best practices by Tom Ryder

May 08, 2016 | sanctum.geek.nz

The time-based job scheduler cron(8) has been around since Version 7 Unix, and its crontab(5) syntax is familiar even for people who don't do much Unix system administration. It's standardised , reasonably flexible, simple to configure, and works reliably, and so it's trusted by both system packages and users to manage many important tasks.

However, like many older Unix tools, cron(8) 's simplicity has a drawback: it relies upon the user to know some detail of how it works, and to correctly implement any other safety checking behaviour around it. Specifically, all it does is try and run the job at an appropriate time, and email the output. For simple and unimportant per-user jobs, that may be just fine, but for more crucial system tasks it's worthwhile to wrap a little extra infrastructure around it and the tasks it calls.

There are a few ways to make the way you use cron(8) more robust if you're in a situation where keeping track of the running job is desirable.

Apply the principle of least privilege

The sixth column of a system crontab(5) file is the username of the user as which the task should run:

0 * * * *  root  cron-task

To the extent that is practical, you should run the task as a user with only the privileges it needs to run, and nothing else. This can sometimes make it worthwhile to create a dedicated system user purely for running scheduled tasks relevant to your application.

0 * * * *  myappcron  cron-task

This is not just for security reasons, although those are good ones; it helps protect you against nasties like scripting errors attempting to remove entire system directories .

Similarly, for tasks with database systems such as MySQL, don't use the administrative root user if you can avoid it; instead, use or even create a dedicated user with a unique random password stored in a locked-down ~/.my.cnf file, with only the needed permissions. For a MySQL backup task, for example, only a few permissions should be required, including SELECT , SHOW VIEW , and LOCK TABLES .

In some cases, of course, you really will need to be root . In particularly sensitive contexts you might even consider using sudo(8) with appropriate NOPASSWD options, to allow the dedicated user to run only the appropriate tasks as root , and nothing else.

Test the tasks

Before placing a task in a crontab(5) file, you should test it on the command line, as the user configured to run the task and with the appropriate environment set. If you're going to run the task as root , use something like su or sudo -i to get a root shell with the user's expected environment first:

$ sudo -i -u cronuser
$ cron-task

Once the task works on the command line, place it in the crontab(5) file with the timing settings modified to run the task a few minutes later, and then watch /var/log/syslog with tail -f to check that the task actually runs without errors, and that the task itself completes properly:

May  7 13:30:01 yourhost CRON[20249]: (you) CMD (cron-task)

This may seem pedantic at first, but it becomes routine very quickly, and it saves a lot of hassles down the line as it's very easy to make an assumption about something in your environment that doesn't actually hold in the one that cron(8) will use. It's also a necessary acid test to make sure that your crontab(5) file is well-formed, as some implementations of cron(8) will refuse to load the entire file if one of the lines is malformed.

If necessary, you can set arbitrary environment variables for the tasks at the top of the file:

MYVAR=myvalue

0 * * * *  you  cron-task
Don't throw away errors or useful output

You've probably seen tutorials on the web where in order to keep the crontab(5) job from sending standard output and/or standard error emails every five minutes, shell redirection operators are included at the end of the job specification to discard both the standard output and standard error. This kluge is particularly common for running web development tasks by automating a request to a URL with curl(1) or wget(1) :

*/5 * * *  root  curl https://example.com/cron.php >/dev/null 2>&1

Ignoring the output completely is generally not a good idea, because unless you have other tasks or monitoring ensuring the job does its work, you won't notice problems (or know what they are), when the job emits output or errors that you actually care about.

In the case of curl(1) , there are just way too many things that could go wrong, that you might notice far too late:

The author has seen all of the above happen, in some cases very frequently.

As a general policy, it's worth taking the time to read the manual page of the task you're calling, and to look for ways to correctly control its output so that it emits only the output you actually want. In the case of curl(1) , for example, I've found the following formula works well:

curl -fLsS -o /dev/null http://example.com/

This way, the curl(1) request should stay silent if everything is well, per the old Unix philosophy Rule of Silence .

You may not agree with some of the choices above; you might think it important to e.g. log the complete output of the returned page, or to fail rather than silently accept a 301 redirect, or you might prefer to use wget(1) . The point is that you take the time to understand in more depth what the called program will actually emit under what circumstances, and make it match your requirements as closely as possible, rather than blindly discarding all the output and (worse) the errors. Work with Murphy's law ; assume that anything that can go wrong eventually will.

Send the output somewhere useful

Another common mistake is failing to set a useful MAILTO at the top of the crontab(5) file, as the specified destination for any output and errors from the tasks. cron(8) uses the system mail implementation to send its messages, and typically, default configurations for mail agents will simply send the message to an mbox file in /var/mail/$USER , that they may not ever read. This defeats much of the point of mailing output and errors.

This is easily dealt with, though; ensure that you can send a message to an address you actually do check from the server, perhaps using mail(1) :

$ printf '%s\n' 'Test message' | mail -s 'Test subject' [email protected]

Once you've verified that your mail agent is correctly configured and that the mail arrives in your inbox, set the address in a MAILTO variable at the top of your file:

[email protected]

0 * * * *    you  cron-task-1
*/5 * * * *  you  cron-task-2

If you don't want to use email for routine output, another method that works is sending the output to syslog with a tool like logger(1) :

0 * * * *   you  cron-task | logger -it cron-task

Alternatively, you can configure aliases on your system to forward system mail destined for you on to an address you check. For Postfix, you'd use an aliases(5) file.

I sometimes use this setup in cases where the task is expected to emit a few lines of output which might be useful for later review, but send stderr output via MAILTO as normal. If you'd rather not use syslog , perhaps because the output is high in volume and/or frequency, you can always set up a log file /var/log/cron-task.log but don't forget to add a logrotate(8) rule for it!

Put the tasks in their own shell script file

Ideally, the commands in your crontab(5) definitions should only be a few words, in one or two commands. If the command is running off the screen, it's likely too long to be in the crontab(5) file, and you should instead put it into its own script. This is a particularly good idea if you want to reliably use features of bash or some other shell besides POSIX/Bourne /bin/sh for your commands, or even a scripting language like Awk or Perl; by default, cron(8) uses the system's /bin/sh implementation for parsing the commands.

Because crontab(5) files don't allow multi-line commands, and have other gotchas like the need to escape percent signs % with backslashes, keeping as much configuration out of the actual crontab(5) file as you can is generally a good idea.

If you're running cron(8) tasks as a non-system user, and can't add scripts into a system bindir like /usr/local/bin , a tidy method is to start your own, and include a reference to it as part of your PATH . I favour ~/.local/bin , and have seen references to ~/bin as well. Save the script in ~/.local/bin/cron-task , make it executable with chmod +x , and include the directory in the PATH environment definition at the top of the file:

PATH=/home/you/.local/bin:/usr/local/bin:/usr/bin:/bin
[email protected]

0 * * * *  you  cron-task

Having your own directory with custom scripts for your own purposes has a host of other benefits, but that's another article

Avoid /etc/crontab

If your implementation of cron(8) supports it, rather than having an /etc/crontab file a mile long, you can put tasks into separate files in /etc/cron.d :

$ ls /etc/cron.d
system-a
system-b
raid-maint

This approach allows you to group the configuration files meaningfully, so that you and other administrators can find the appropriate tasks more easily; it also allows you to make some files editable by some users and not others, and reduces the chance of edit conflicts. Using sudoedit(8) helps here too. Another advantage is that it works better with version control; if I start collecting more than a few of these task files or to update them more often than every few months, I start a Git repository to track them:

$ cd /etc/cron.d
$ sudo git init
$ sudo git add --all
$ sudo git commit -m "First commit"

If you're editing a crontab(5) file for tasks related only to the individual user, use the crontab(1) tool; you can edit your own crontab(5) by typing crontab -e , which will open your $EDITOR to edit a temporary file that will be installed on exit. This will save the files into a dedicated directory, which on my system is /var/spool/cron/crontabs .

On the systems maintained by the author, it's quite normal for /etc/crontab never to change from its packaged template.

Include a timeout

cron(8) will normally allow a task to run indefinitely, so if this is not desirable, you should consider either using options of the program you're calling to implement a timeout, or including one in the script. If there's no option for the command itself, the timeout(1) command wrapper in coreutils is one possible way of implementing this:

0 * * * *  you  timeout 10s cron-task

Greg's wiki has some further suggestions on ways to implement timeouts .

Include file locking to prevent overruns

cron(8) will start a new process regardless of whether its previous runs have completed, so if you wish to avoid locking for long-running task, on GNU/Linux you could use the flock(1) wrapper for the flock(2) system call to set an exclusive lockfile, in order to prevent the task from running more than one instance in parallel.

0 * * * *  you  flock -nx /var/lock/cron-task cron-task

Greg's wiki has some more in-depth discussion of the file locking problem for scripts in a general sense, including important information about the caveats of "rolling your own" when flock(1) is not available.

If it's important that your tasks run in a certain order, consider whether it's necessary to have them in separate tasks at all; it may be easier to guarantee they're run sequentially by collecting them in a single shell script.

Do something useful with exit statuses

If your cron(8) task or commands within its script exit non-zero, it can be useful to run commands that handle the failure appropriately, including cleanup of appropriate resources, and sending information to monitoring tools about the current status of the job. If you're using Nagios Core or one of its derivatives, you could consider using send_nsca to send passive checks reporting the status of jobs to your monitoring server. I've written a simple script called nscaw to do this for me:

0 * * * *  you  nscaw CRON_TASK -- cron-task
Consider alternatives to cron(8)

If your machine isn't always on and your task doesn't need to run at a specific time, but rather needs to run once daily or weekly, you can install anacron and drop scripts into the cron.hourly , cron.daily , cron.monthly , and cron.weekly directories in /etc , as appropriate. Note that on Debian and Ubuntu GNU/Linux systems, the default /etc/crontab contains hooks that run these, but they run only if anacron(8) is not installed.

If you're using cron(8) to poll a directory for changes and run a script if there are such changes, on GNU/Linux you could consider using a daemon based on inotifywait(1) instead.

Finally, if you require more advanced control over when and how your task runs than cron(8) can provide, you could perhaps consider writing a daemon to run on the server consistently and fork processes for its task. This would allow running a task more often than once a minute, as an example. Don't get too bogged down into thinking that cron(8) is your only option for any kind of asynchronous task management!

[Oct 31, 2017] Bash job control by Tom Ryder

Jan 31, 2012 | sanctum.geek.nz

Oftentimes you may wish to start a process on the Bash shell without having to wait for it to actually complete, but still be notified when it does. Similarly, it may be helpful to temporarily stop a task while it's running without actually quitting it, so that you can do other things with the terminal. For these kinds of tasks, Bash's built-in job control is very useful. Backgrounding processes

If you have a process that you expect to take a long time, such as a long cp or scp operation, you can start it in the background of your current shell by adding an ampersand to it as a suffix:

$ cp -r /mnt/bigdir /home &
[1] 2305

This will start the copy operation as a child process of your bash instance, but will return you to the prompt to enter any other commands you might want to run while that's going.

The output from this command shown above gives both the job number of 1, and the process ID of the new task, 2305. You can view the list of jobs for the current shell with the builtin jobs :

$ jobs
[1]+  Running  cp -r /mnt/bigdir /home &

If the job finishes or otherwise terminates while it's backgrounded, you should see a message in the terminal the next time you update it with a newline:

[1]+  Done  cp -r /mnt/bigdir /home &
Foregrounding processes

If you want to return a job in the background to the foreground, you can type fg :

$ fg
cp -r /mnt/bigdir /home &

If you have more than one job backgrounded, you should specify the particular job to bring to the foreground with a parameter to fg :

$ fg %1

In this case, for shorthand, you can optionally omit fg and it will work just the same:

$ %1
Suspending processes

To temporarily suspend a process, you can press Ctrl+Z:

$ cp -r /mnt/bigdir /home
^Z
[1]+  Stopped  cp -r /mnt/bigdir /home

You can then continue it in the foreground or background with fg %1 or bg %1 respectively, as above.

This is particularly useful while in a text editor; instead of quitting the editor to get back to a shell, or dropping into a subshell from it, you can suspend it temporarily and return to it with fg once you're ready.

Dealing with output

While a job is running in the background, it may still print its standard output and standard error streams to your terminal. You can head this off by redirecting both streams to /dev/null for verbose commands:

$ cp -rv /mnt/bigdir /home &>/dev/null

However, if the output of the task is actually of interest to you, this may be a case where you should fire up another terminal emulator, perhaps in GNU Screen or tmux , rather than using simple job control.

Suspending SSH sessions

As a special case, you can suspend an SSH session using an SSH escape sequence . Type a newline followed by a ~ character, and finally press Ctrl+Z to background your SSH session and return to the terminal from which you invoked it.

tom@conan:~$ ssh crom
tom@crom:~$ ~^Z [suspend ssh]
[1]+  Stopped  ssh crom
tom@conan:~$

You can then resume it as you would any job by typing fg :

tom@conan:~$ fg %1
ssh crom
tom@crom:~$

[Sep 29, 2017] Writing Recurring Scripts

...The crontab (chronological table) command maintains a list of jobs for cron to execute. Each user has his or her own crontab table. The -l (list) switch lists currently scheduled tasks. Linux reports an error if you don't have permission to use cron. Because jobs are added or removed from the crontab table as a group, always start with the -l switch, saving the current table to a file.

$ crontab -l > cron.txt
				

After the current table is saved, the file can be edited. There are five columns for specifying the times when a program is to run: The minute, hour, day, month, and the day of the week. Unused columns are marked with an asterisk, indicating any appropriate time.

Times are represented in a variety of formats: Individually (1), comma-separated lists (1,15), ranges (0-6, 9-17), and ranges with step values (1-31/2). Names can be used for months or days of the week.

The final column contains the name of the command to execute. The following line runs a script called cleanup.sh at 1:00 AM every morning.

*       1       *       *       *       /home/kburtch/cleanup.sh

Environment variables can also be initialized in the crontab. When a shell script is started by cron, it is not started from a login session and none of the profile files are executed. Only a handful of variables are defined: PWD, HOSTNAME, MACHTYPE, LOGNAME, SHLVL, SHELL, HOSTTYPE, OSTYPE, HOME, TERM, and PATH. You have to explicitly set any other values in the script or in the crontab list.

PATH is defined as only /usr/bin:/bin. Other paths are normally added by profile files and so are unavailable.

Because a script running under cron is not in a login session, there is no screen to write standard output to. Anything that is normally written to standard output is instead captured by cron and mailed to the account owning the cron script. The mail has the unhelpful subject line of cron. Even printing a blank line results in a seemingly empty email being sent. For this reason, scripts designed to run under cron should either write their output to a log file, or should create and forward their own email with a meaningful subject line. It is common practice to write a wrapper script to capture the output from the script doing the actual work.

#!/bin/bash
#
# show_users.sh: show all users in the database table "users"

shopt -s -o nounset

declare -rx SCRIPT=${0##*/}
declare -r SQL_CMDS="sort_inventory.sql"
declare -rx ON_ERROR_STOP

if [ ! -r "$SQL_CMDS" ] ; then
   printf "$SCRIPT: the SQL script $SQL_CMDS doesn't exist or is not \
 readable" >&2
   exit 192
fi

RESULTS=`psql -user gordon -dbname custinfo  –quiet -no-align -tuples-only \
 -field-separator "," -file "$SQL_CMDS"`
if [ $? -ne 0 ] ; then
   printf "$SCRIPT: SQL statements failed." >&2
   exit 192
fi 


					  
#!/bin/bash
# show_users_wrapper.sh - show_users.sh wrapper script

shopt -s -o nounset

declare -rx SCRIPT=${0##*/}
declare -rx USER="kburtch"
declare -rx mail="/bin/mail"
declare -rx OUTPUT='mktemp /tmp/script_out.XXXXXX'
declare -rx SCRIPT2RUN="./show_users.sh"

# sanity checks

if test ! -x "$mail" ; then
   printf "$SCRIPT:$LINENO: the command $mail is not available - aborting" >&2
   exit 1
fi

if test ! -x "$SCRIPT2RUN" ; then
   printf "$SCRIPT: $LINENO: the command $SCRIPT2RUN is not available\
 - aborting" >&2
   exit 1
fi

# record the date for any errors, and create the OUTPUT file

date > $OUTPUT

# run the script

$SCRIPT2RUN 2>&1 > "$OUTPUT"

# mail errors to USER

if [ $? -ne 0 ] ; then
   $mail -s "$SCRIPT2RUN failed" "$USER" < "$OUTPUT"
fi

# cleanup

rm "$OUTPUT"
exit 0
				

					  

[Apr 17, 2014] CronHowto - Community Help Wiki

It is possible to run gui applications via cronjobs. This can be done by telling cron which display to use.

00 06 * * * env DISPLAY=:0 gui_appname

The env DISPLAY=:0 portion will tell cron to use the current display (desktop) for the program "gui_appname".

And if you have multiple monitors, don't forget to specify on which one the program is to be run. For example, to run it on the first screen (default screen) use :

00 06 * * * env DISPLAY=:0.0 gui_appname

The env DISPLAY=:0.0 portion will tell cron to use the first screen of the current display for the program "gui_appname".

Note: GUI users may prefer to use gnome-schedule (aka "Scheduled tasks") to configure GUI cron jobs. In gnome-schedule, when editing a GUI task, you have to select "X application" in a dropdown next to the command field.

Note: In Karmic(9.10), you have to enable X ACL for localhost to connect to for GUI applications to work.

 ~$ xhost +local:
non-network local connections being added to access control list
 ~$ xhost
access control enabled, only authorized clients can connect
LOCAL:
...

Tips

crontab -e uses the EDITOR environment variable. to change the editor to your own choice just set that. You may want to set EDITOR in you .bashrc because many commands use this variable. Let's set the EDITOR to nano a very easy editor to use:

export EDITOR=nano

There are also files you can edit for system-wide cron jobs. The most common file is located at /etc/crontab, and this file follows a slightly different syntax than a normal crontab file. Since it is the base crontab that applies system-wide, you need to specify what user to run the job as; thus, the syntax is now:

minute(s) hour(s) day(s)_of_month month(s) day(s)_of_week user command

It is recommended, however, that you try to avoid using /etc/crontab unless you need the flexibility offered by it, or if you'd like to create your own simplified anacron-like system using run-parts for example. For all cron jobs that you want to have run under your own user account, you should stick with using crontab -e to edit your local cron jobs rather than editting the system-wide /etc/crontab.

Crontab Example

Below is an example of how to setup a crontab to run updatedb, which updates the slocate database: Open a term, type "crontab -e" (without the double quotes) and press enter. Type the following line, substituting the full path of the application you wish to run for the one shown below, into the editor:

45 04 * * * /usr/bin/updatedb

Save your changes and exit the editor.

Crontab will let you know if you made any mistakes. The crontab will be installed and begin running if there are no errors. That's it. You now have a cronjob setup to run updatedb, which updates the slocate database, every morning at 4:45.

Note: The double-ampersand (&&) can also be used in the "command" section to run multiple commands consecutively, but only if the previous command exits successfully. A string of commands joined by the double-ampersand will only get to the last command if all the previous commands are run successfully. If exit error-checking is not of a concern, string commands together, separated with a semi-colon (;)

45 04 * * * /usr/sbin/chkrootkit && /usr/bin/updatedb

The above example will run chkrootkit followed by updatedb at 4:45am daily - providing you have all listed apps installed. If chkrootkit fails, updatedb will NOT be run.

[Oct 17, 2013] Crontab file

The UNIX and Linux Forums

mradsus

Hi All,
I created a crontab entry in a cron.txt file accidentally entered

crontab cron.txt.

Now my previous crontab -l entries are not showing up, that means i removed the scheduling of the previous jobs by running this command "crontab cron.txt"

How do I revert back to previously schedule jobs.
Please help. this is urgent.,
Thanks.

lestat_ecuador:
Try to go to /tmp and do
Code:
ls -ltr cron*
Then will appear kind a history of the cron, see the creation dates an look into the one you want. For example I have:
Code:
usuario: > cd /tmp/
usuario: /tmp > ls -ltr cro*
-rw-------   1 ecopge   ecuador      859 Jul 25 08:33 crontabJhaiKP
-rw-------   1 ecppga   ecuador        0 Jul 28 16:00 croutFNZCuVqsb

I already modify my crontab file, it is into croutFNZCuVqsb but my last crontab file is into [B]crontabJhaiKP[B].

try it and let me know how are you going.

[Oct 17, 2013] How to Recover User crontab -r

House of Linux

This is a quick advice of those who have ran crontab -r accidentally and would like to restore it. Well letter r and letter e is close together therefore this is a very easy mistake to do.

If you don't have a backup of /var/spool/cron/*user* the only way you could find to recover you crontab commands would be taking a look at the logs of your system.

For example in RedHat / CentOS Linux distributions you can issue: # cat /var/log/cron And you will see a list of commands executed and the time, from there you will be able to rebuild your crontab.

Hope this will help someone...

[Mar 22, 2011] incron-cron-inotify

Unfortunately incron doesn't have recursive auto subscriptions, i.e. if I want to watch an entire tree and automatically subscribe to new directories being created. look at lsyncd, although it is primarily meant as a syncing tool, you can configure any action on an event.

incron is very similar in concept and usage to using cron, as the interface is a clone of it.

Each user who is allowed to use incron may use the incrontab command to view, or edit, their rule list. These rules are processed via the daemon, and when a match occurs the relevant command is executed.

To list the current rules you've got defined run "incrontab -l", and to edit them use "incrontab -e". If you do that just now you'll receive the following error message:

rt:~# incrontab  -l
user 'root' is not allowed to use incron

This error may be fixed in one of two ways:

Allow the root user to make use of incron: By editing /etc/incron.allow, adding 'root' to it.

Allowing all local users the ability to use incron: By removing the file /etc/incron.allow.

The user table rows have the following syntax (use one or more spaces between elements):

[Path] [mask] [command]

Where:

The full list of supported flags for mask include:

The mask may additionally contain a special symbol IN_NO_LOOP which disables events occurred during processing the event (to avoid loops).

The command may contain these wildcards:

Example Usage

/tmp/spool IN_CLOSE_WRITE /usr/local/bin/run-spool $@/$#

This says "Watch /tmp/spool, and when an IN_CLOSE_WRITE event occurs run /usr/local/bin/run-spool with the name of the file that was created".

Create your backups

This small script backup all files in the etc and myProject directory.

 vi /root/inotify.sh

--

 #!/bin/sh

 # Create a inotify backup dir (if not exists)
 #
 mkdir /var/backups/inotify

 # Make a copy off the full path and file
 #
 cp -p --parents $1  /var/backups/inotify

 # move the file to a file with datetime-stamp
 #
 mv /var/backups/inotify$1 /var/backups/inotify$1_`date +'%Y-%m-%d_%H:%M'

Make the file executable for root

 chmod 755 /root/inotify.sh

Open

 incrontab -e

And add:

 /etc IN_CLOSE_WRITE,IN_MODIFY /root/inotify.sh $@/$#
 /home/andries/myProject IN_CLOSE_WRITE /root/inotify.sh $@/$#

So every time a file is wrote in the watched directory it's also saved in the given directory

Selected Comments

joe :

I am not sure if your script will work as intended

1. mkdir -p /var/backups/inotify The -p will make sure that the dir is created when it not exists

2. cp -p –parents $1 /var/backups/inotify i have no idea why you need parents but -a (archive) may be more useful

2a. make use of the cp backup facility cp –backup=numbered will make simply number your backups automatically.

[Mar 21, 2011] Gentoo Linux cron Guide

Good tutorial
Gentoo Linux Documentation

Troubleshooting

If you're having problems getting cron to work properly, you might want to go through this quick checklist.

[Jan 30, 2010] Cron - cron-like scheduler for Perl subroutines

This module provides a simple but complete cron like scheduler. I.e this modules can be used for periodically executing Perl subroutines. The dates and parameters for the subroutines to be called are specified with a format known as crontab entry (METHODS, add_entry() and crontab(5))

The philosophy behind Schedule::Cron is to call subroutines periodically from within one single Perl program instead of letting cron trigger several (possibly different) perl scripts. Everything under one roof. Furthermore Schedule::Cron provides mechanism to create crontab entries dynamically, which isn't that easy with cron.

Schedule::Cron knows about all extensions (well, at least all extensions I'm aware of, i.e those of the so called "Vixie'' cron) for crontab entries like ranges including 'steps', specification of month and days of the week by name or coexistence of lists and ranges in the same field. And even a bit more (like lists and ranges with symbolic names).

[Jan 27, 2010] Using at (@) and Percentage (%) in Crontab

There is an easy way to start a program during system boot. Just put this in your crontab:
@reboot /path/to/my/program
The command will be executed on every (re)boot. Crontab can be modified by running
#crontab -e
Other available Options
string meaning
------ -----------
@reboot Run once, at startup.
@yearly Run once a year, "0 0 1 1 *".
@annually (same as @yearly)
@monthly Run once a month, "0 0 1 * *".
@weekly Run once a week, "0 0 * * 0".
@daily Run once a day, "0 0 * * *".
@midnight (same as @daily)
@hourly Run once an hour, "0 * * * *"
More information about crontab options is available in the man page check here

How to Use percentage sign (%) in a crontab entry

Usually, a % is used to denote a new line in a crontab entry. The first % is special in that it denotes the start of STDIN for the crontab entry's command. A trivial example is:
* * * * * cat - % another minute has passed
This would output the text
another minute has passed
After the first %, all other %s in a crontab entry indicate a new line. So a slightly different trivial example is:
* * * * * cat - % another % minute % has % passed 
This would output the text
another
minute
has
passed
Note how the % has been used to indicate a new line.

The problem is how to use a % in a crontab line to as a % and not as a new line. Many manuals will say escape it with a \. This certainly stops its interpretation as a new line but the shell running the cron job can leave the \ in. For example:

* * * * * echo '\% another \% minute \% has \% passed'
would output the text
\% another \% minute \% has \% passed
Clearly, not what was intended.

A solution is to pass the text through sed. The crontab example now becomes:

* * * * * echo '\% another \% minute \% has \% passed'| sed -e 's|\\||g'
This would output the text
% another % minute % has % passed
which is what was intended.

This technique is very useful when using a MySQL command within a crontab. MySQL command can often have a % in them. Some example are:

SET @monyy=DATE_FORMAT(NOW(),"%M %Y")
SELECT * FROM table WHERE name LIKE 'fred%'
So, to have a crontab entry to run the MySQL command
mysql -vv -e "SELECT * FROM table WHERE name LIKE Fred%'" member_list
would have to appear in the crontab as
echo "SELECT * FROM table WHERE name LIKE 'Fred\%'" | sed -e 's|\\||g' | mysql -vv member_list
Pulling the crontab entry apart there is:
the echo command sends the MySQL command to STDOUT where it is piped into
sed which removes any back slashes before sending the output to STDOUT where it is piped into
the mysql command processor which reads its commands from STDIN

Pycron

Windows-compatible implementation of cron in Python
This article will discuss using a Cron type system, as used on Unix and Linux systems, to bring the flexibility, scalability and a need for more out of a task automation tool, to the Win32 environment.

Internals: Replacing Task Scheduler with Cron

As our dependency upon machines for various tasks grow, time becomes an integral factor that may work against us, when pressed upon deadlines. Automation of such tasks, without the need for human intervention, becomes vital to whether one is able to square away enough time to complete more and more tasks with little help from human hands.

Task Scheduler, which comes bundled with Windows attempts to make automation of tasks effortless. Unfortunately, it is not very configurable and basic in what it is capable of.

On Unix and Linux systems, Cron is what is used for task scheduling. This scheduler is very configurable, and is capable of well more then its Windows counterpart.

This article will discuss using a Cron type system, as used on Unix and Linux systems, to bring the flexibility, scalability and a need for more out of a task automation tool, to the Win32 environment.

Tools:

Pycron from kalab.com

Install the product and make sure to install it as a service (default option on the last dialog box of the installer) if your Win32 operating system supports this.

Then click Start->Run->services.msc (hit enter)

Scroll down to and highlight Task Scheduler->right click->Properties->Toggle to Manual->hit Stop->then Apply->OK

Then scroll up to Python Cron Service->highlight->right click->Properties->Toggle to Automatic->Apply->OK

We will be working with a file called crontab.txt for all of the scheduled entries. This file must be created in the Pycron Program directory which is located in the pycron folder under the Program Files folder.

Create a new file called crontab.txt in the Pycron Program Directory and put this in to it

* * * * * replace replace
Save your file.

Now launch the crontab editor (Start->Programs->Pycron->Pycron CronTab Editor)

By default it will load up the contents of the crontab.txt file in the Pycron Program Directory.

Screenshot

The parameters of a crontab entry from left to right are as follows.

0-59 - Minute
0-23 - Hour
1-31 - Day
1-12 - Month
0-6 - Day of the week (0 For Mon, and 6 for Sunday)

Command/Application to execute

Parameter to the application/command.

entry:
Minute Hour Day Month Day_of_the_week Command Parameter

Note:
* is a wildcard and matches all values
/ is every (ex: every 10 minutes = */10) (new in this Win32 version)
, is execute each value (ex: every 10 minutes = 0,10,20,30,40,50)
- is to execute each value in the range (ex: from 1st (:01) to 10th (:10) minute = 1-10)

Double click the 1st entry of " * * * * replace replace" to edit the entry.

Screenshot

For an example, we will run defrag on every Friday at 11:00 (23:00) PM against the C:\ volume.

On the Command line hit Browse, and navigate to your System32 Folder inside your Windows folder and double click on defrag.exe

On the Parameter line enter in c:

Always run a Test Execution to make sure your command is valid. If all was successful, you will see your command/application run and a kick back message of Successful will be observed.

For Minute, erase the * and enter in 0
For Hour, erase the * and enter in 23
For Week Day enter in 4.

Then hit OK, File->Save.

Note: You can use the wizard to enter in values for each entry as well.

Now open up a command prompt (start->run->cmd), and type:
net start pycron

You can leave it running now, and every time you append and or change your crontab.txt, the schedule will be updated.

To add another entry using the crontab GUI, add in a * * * * * replace replace to crontab.txt on the next free line, save it, then open up crontab.txt with the GUI editor and make the desired changes on the entry by double clicking to edit.

It is recommended that every time the GUI is used to edit entries, that you observe the entry in crontab. After you become comfortable with the syntax of cron entries, there will be no need for the GUI editor.

The entry for our defrag command becomes:
0 23 * * 4 "C:\WINDOWS\System32\defrag.exe" c:

This same task can be performed with the Task Scheduler with one entry.

Let us go through another example of more complexity, which would not as easily be accomplished with the Task Scheduler.

I want to back up my work files every 3 hours, from Monday through Friday, between the hours of 9AM to 6PM, for all the months of the year. The work folder is C:\WORK and the backup folder is C:\BACKUP.

Open up crontab.txt and on the next free line, enter in * * * * * replace replace, then save it.

Open up the crontab editor and import crontab.txt. Double click the "* * * * replace replace" entry.

For Command, browse to xcopy located in System32 within your Windows folder.

For Parameter: C:\WORK\* C:\BACKUP /Y

For Minute: 0
For Hour: 9-18/3
For Day: *
For Month: *
For Week Day: 0-4

Click OK->File->Save.

The entry for this task as reflected in our crontab.txt becomes
0 9-18/3 * * 0-4 "C:\WINDOWS\System32\xcopy.exe" C:\WORK\* C:\BACKUP /Y

If we were to schedule the above example with the Task Scheduler that comes with windows, then a separate entry for every 3rd hour mark in terms of time (AM/PM) between the aforementioned times would have to be entered, for the task.

Note: Cron can work with your own written batch/script files as well.

Note: You can view other examples in crontab.sample, located in the pycron program directory.

As you can see, Cron has a lot more to offer then the Task Scheduler. Not to say that the Windows application is not useable, but for those scenarios where you need the ability to be flexible, and configurable without all the extra headaches, then this is the ideal replacement for you. It proves to be much more efficient as well in practice.

Further Reading:

Cron Help Guide at linuxhelp.net

Pycron Home

OK to change time of cron.daily

Linux Forums
Scripts in /etc/cron.daily run each day from 4:02 to 0:02. I want do this so that updatedb (started by slocate.cron) finishes before people start their workday. My quesiton isn't how, but whether this will create any problems.

Does anyone see any problems with running cron.daily at 2 minutes after midnight instead of 2 minutes after 4?

The tasks that run are mostly cleaning up log files and deleting old maid Hat EL. The daily tasks are all defaults. (ls /etc/cron.daily returns: 00-logwatch 00webalizer certwatch logrotate makewhatis.cron prelink rpm slocate.cron tetex.cron tmpwatch).

Thanks in advance!

===

Originally Posted by Tim65

Does anyone see any problems with running cron.daily at 2 minutes after midnight instead of 2 minutes after 4?

This should be no problem.

However: if you need to apply any patches / security fixes to cron in the future, you will want to confirm that your changes weren't overwritten.

===

n.yaghoobi.s : Thanks. I know how to change it - I was just wondering if anyone thought it was a bad idea.

anomie : Thanks. Good point about confirming the change doesn't get undone by patches. I'm going to make the change. If I ever do discover a problem caused by this change, I'll be sure to look up this thread and post the info here.

[Jun 16, 2008] Cron Sandbox

CGI script that allow to enter a crontab command and produce forward schedule of times when it will run for testing.

[May 14, 2007] Neat crontab tricks lxpages.com blog

Linux only shortcuts.

There are several special entries, some which are just shortcuts, that you can use instead of specifying the full cron entry. The most useful of these is probably @reboot which allows you to run a command each time the computer gets reboot. You can alert yourself when server is back online after a reboot. Also becomes useful if you want to run certain services or commands at start up. The complete list of special entries are:

@monthly
Entry Description Equivalent To
@reboot Run once, at startup. None Run once a month 0 0 1 * *
@weekly Run once a week 0 0 * * 0
@daily Run once a day 0 0 * * *
@midnight (same as @daily) 0 0 * * *
@hourly Run once an hour 0 * * * *

The most useful again is @reboot. Use it to notify you when your server gets rebooted!

crontab

Users are permitted to use crontab if their names appear in the file /usr/lib/cron/cron.allow. If that file does not exist, the file /usr/lib/cron/cron.deny is checked to determine if the user should be denied access to crontab. If neither file exists, only a process with appropriate privileges is allowed to submit a job. If only cron.deny exists and is empty, global usage is permitted. The cron.allow and cron.deny files consist of one user name per line.

Sun_Solaris_AuditGuide

if both cron.allow and cron.deny files exist the cron.deny is ignored.

This can be accomplished by either listing users permitted to use the command in the file /var/spool/cron/cron.allow and the /var/spool/cron/at.allow or in the list of user not permitted to access the command in the file /var/spool/cron/cron.deny

Linux tip Controlling the duration of scheduled jobs

A very god article with a lot of examples

[ian@attic4 ~]$ cat ./runclock3.sh #!/bin/bash runtime=${1:-10m} mypid=$$ # Run xclock in background xclock& clockpid=$! echo "My PID=$mypid. Clock's PID=$clockpid" ps -f $clockpid #Sleep for the specified time. sleep $runtime kill -s SIGTERM $clockpid echo "All done"

Listing 5 shows what happens when you execute runclock3.sh. The final kill command confirms that the xclock process (PID 9285) was, indeed, terminated.


Listing 5. Verifying the termination of child processes
             
[ian@attic4 ~]$ ./runclock3.sh 5s
My PID=9284. Clock's PID=9285
UID        PID  PPID  C STIME TTY      STAT   TIME CMD
ian       9285  9284  0 22:14 pts/1    S+     0:00 xclock
All done
[ian@attic4 ~]$ kill -0 9285
bash: kill: (9285) - No such process
If you omit the signal specification, then SIGTERM is the default signal. The SIG part of a signal name is optional. Instead of using -s and a signal name, you can just prefix the signal number with a -, so the four forms shown in Listing 6 are equivalent ways of killing process 9285. Note that the special value -0, as used in Listing 4 above, tests whether a signal could be sent to a process.

Listing 6. Ways to specify signals with the kill command

             
kill -s SIGTERM 9285
kill -s TERM 9285
kill -15 9285
kill 9285

If you need just a one-shot timer to drive an application, such as you have just seen here, you might consider the timeout command, which is part of the AppleTalk networking package (Netatalk). You may need to install this package (see Resources below for details), since most installations do not include it automatically.

Other termination conditions

You now have the basic tools to run a process for a fixed amount of time. Before going deeper into signal handling, let's consider how to handle other termination requirements, such as repetitively capturing information for a finite time, terminating when a file becomes a certain size, or terminating when a file contains a particular string. This kind of work is best done using a loop, such as for, while, or until, with the loop executed repeatedly with some built-in delay provided by the sleep command. If you need finer granularity than seconds, you can also use the usleep command.

You can add a second hand to the clock, and you can customize colors. Use the showrgb command to explore available color names. Suppose you use the command xclock -bg Thistle -update 1& to start a clock with a second hand, and a Thistle-colored background.

Now you can use a loop with what you have learned already to capture images of the clock face every second and then combine the images to make an animated GIF image. Listing 7 shows how to use the xwininfo command to find the window id for the xclock command. Then use ImageMagick command-line tools to capture 60 clock face images at one-second intervals (see Resources for details on ImageMagick). And finally combine these into an infinitely looping animated GIF file that is 50% of the dimensions of the original clock.


Listing 7. Capturing images one second apart
             
[ian@attic4 ~]$ cat getclock.sh
#!/bin/bash
windowid=$(xwininfo -name "xclock"| grep '"xclock"' | awk '{ print $4 }')
sleep 5
for n in `seq 10 69`; do
  import -frame  -window $windowid clock$n.gif&
  sleep 1s
#  usleep 998000
done
convert -resize 50% -loop 0 -delay 100 clock?[0-9].gif clocktick.gif
[ian@attic4 ~]$ ./getclock.sh
[ian@attic4 ~]$ file clocktick.gif
clocktick.gif: GIF image data, version 89a, 87 x 96

Timing of this type is always subject to some variation, so the import command to grab the clock image is run in the background, leaving the main shell free to keep time. Nevertheless, some drift is likely to occur because it does take a finite amount of time to launch each subshell for the background processing. This example also builds in a 5-second delay at the start to allow the shell script to be started and then give you time to click on the clock to bring it to the foreground. Even with these caveats, some of my runs resulted in one missed tick and an extra copy of the starting tick because the script took slightly over 60 seconds to run. One way around this problem would be to use the usleep command with a number of microseconds that is enough less than one second to account for the overhead, as shown by the commented line in the script. If all goes as planned, your output image should be something like that in Figure 2.


Figure 2. A ticking xclock

This example shows you how to take a fixed number of snapshots of some system condition at regular intervals. Using the techniques here, you can take snapshots of other conditions. You might want to check the size of an output file to ensure it does not pass some limit, or check whether a file contains a certain message, or check system status using a command such as vmstat. Your needs and your imagination are the only limits.

Signals and traps

If you run the getclock.sh script of Listing 7 yourself, and you close the clock window while the script is running, the script will continue to run but will print error messages each time it attempts to take a snapshot of the clock window. Similarly, if you run the runclock3.sh script of Listing 4, and press Ctrl-c in the terminal window where the script is running, the script will immediately terminate without shutting down the clock. To solve these problems, your script needs to be able to catch or trap some of the signals discussed in Terminating a child process.

If you execute runclock3.sh in the background and run the ps -f command while it is running, you will see output similar to Listing 8.


Listing 8. Process information for runclock3.sh
             
[ian@attic4 ~]$ ./runclock3.sh 20s&
[1] 10101
[ian@attic4 ~]$ My PID=10101. Clock's PID=10102
UID        PID  PPID  C STIME TTY      STAT   TIME CMD
ian      10102 10101  0 06:37 pts/1    S      0:00 xclock
ps -f
UID        PID  PPID  C STIME TTY          TIME CMD
ian       4598 12455  0 Jul29 pts/1    00:00:00 bash
ian      10101  4598  0 06:37 pts/1    00:00:00 /bin/bash ./runclock3.sh 20s
ian      10102 10101  0 06:37 pts/1    00:00:00 xclock
ian      10104 10101  0 06:37 pts/1    00:00:00 sleep 20s
ian      10105  4598  0 06:37 pts/1    00:00:00 ps -f
[ian@attic4 ~]$ All done

[1]+  Done                    ./runclock3.sh 20s

Note that the ps -f output has three entries related to the runclock3.sh process (PID 10101). In particular, the sleep command is running as a separate process. One way to handle premature death of the xclock process or the use of Ctrl-c to terminate the running script is to catch these signals and then use the kill command to kill the sleep command.

There are many ways to accomplish the task of determining the process for the sleep command. Listing 9 shows the latest version of our script, runclock4.sh. Note the following points:


Listing 9. Trapping signals with runclock4.sh
             
[ian@attic4 ~]$ cat runclock4.sh
#!/bin/bash

stopsleep() {
  sleeppid=$1
  echo "$(date +'%T') Awaken $sleeppid!"
  kill -s SIGINT $sleeppid >/dev/null 2>&1
}

runtime=${1:-10m}
mypid=$$
# Enable immediate notification of SIGCHLD
set -bm
# Run xclock in background
xclock&
clockpid=$!
#Sleep for the specified time.
sleep $runtime&
sleeppid=$!
echo "$(date +'%T') My PID=$mypid. Clock's PID=$clockpid sleep PID=$sleeppid"
# Set a trap
trap 'stopsleep $sleeppid' CHLD INT TERM
# Wait for sleeper to awaken
wait $sleeppid
# Disable traps
trap SIGCHLD
trap SIGINT
trap SIGTERM
# Clean up child (if still running)
echo "$(date +'%T') terminating"
kill -s SIGTERM $clockpid >/dev/null 2>&1 && echo "$(date +'%T') Stopping $clockpid"
echo "$(date +'%T') All done"

Listing 10 shows the output from running runclock4.sh three times. The first time, everything runs to its natural completion. The second time, the xclock is prematurely closed. And the third time, the shell script is interrupted with Ctrl-c.


Listing 10. Stopping runclock4.sh in different ways
             
[ian@attic4 ~]$ ./runclock4.sh 20s
09:09:39 My PID=11637. Clock's PID=11638 sleep PID=11639
09:09:59 Awaken 11639!
09:09:59 terminating
09:09:59 Stopping 11638
09:09:59 All done
[ian@attic4 ~]$ ./runclock4.sh 20s
09:10:08 My PID=11648. Clock's PID=11649 sleep PID=11650
09:10:12 Awaken 11650!
09:10:12 Awaken 11650!
[2]+  Interrupt               sleep $runtime
09:10:12 terminating
09:10:12 All done
[ian@attic4 ~]$ ./runclock4.sh 20s
09:10:19 My PID=11659. Clock's PID=11660 sleep PID=11661
09:10:22 Awaken 11661!
09:10:22 Awaken 11661!
09:10:22 Awaken 11661!
[2]+  Interrupt               sleep $runtime
09:10:22 terminating
09:10:22 Stopping 11660
./runclock4.sh: line 31: 11660 Terminated              xclock
09:10:22 All done

Note how many times the stopsleep function is called as evidenced by the "Awaken" messages. If you are not sure why, you might try making a separate copy of this function for each interrupt type that you catch and see what causes the extra calls.

You will also note that some job control messages tell you about termination of the xclock command and interrupting the sleep command. When you run a job in the background with default bash terminal settings, bash normally catches SIGCHLD signals and prints a message after the next terminal output line is printed. The set -bm command in the script tells bash to report SIGCHLD signals immediately and to enable job control monitoring, The alarm clock example in the next section shows you how to suppress these messages.

An alarm clock

Our final exercise returns to the original problem that motivated this article: how to record a radio program. We will actually build an alarm clock. If your laws allow recording of such material for your proposed use, you can build a recorder instead by adding a program such as vsound.

For this exercise, we will use the GNOME rhythmbox application to illustrate some additional points. Even if you use another media player, this discussion should still be useful.

An alarm clock could make any kind of noise you want, including playing your own CDs, or MP3 files. In central North Carolina, we have a radio station, WCPE, that broadcasts classical music 24 hours a day. In addition to broadcasting, WCPE also streams over the Internet in several formats, including Ogg Vorbis. Pick your own streaming source if you prefer something else.

To start rhythmbox from an X Windows terminal session playing the WCPE Ogg Vorbis stream, you use the command shown in Listing 11.


Listing 11. Starting rhythmbox with the WCPE Ogg Vorbis stream
             
rhythmbox --play http://audio-ogg.ibiblio.org:8000/wcpe.ogg

The first interesting point about rhythmbox is that the running program can respond to commands, including a command to terminate. So you don't need to use the kill command to terminate it, although you still could if you wanted to.

The second point is that most media players, like the clock that we have used in the earlier examples, need a graphical display. Normally, you run commands with the cron and at facilities at some point when you may not be around, so the usual assumption is that these scheduled jobs do not have access to a display. The rhythmbox command allows you to specify a display to use. You probably need to be logged on, even if your screen is locked, but you can explore those variations for yourself. Listing 12 shows the alarmclock.sh script that you can use for the basis of your alarm clock. It takes a single parameter, which specifies the amount of time to run for, with a default of one hour.


Listing 12. The alarm clock - alarmclock.sh
             
[ian@attic4 ~]$ cat alarmclock.sh
#!/bin/bash

cleanup () {
  mypid=$1
  echo "$(date +'%T') Finding child pids"
  ps -eo ppid=,pid=,cmd= --no-heading | grep "^ *$mypid"
  ps $playerpid >/dev/null 2>&1 && {
    echo "$(date +'%T') Killing rhythmbox";
    rhythmbox --display :0.0 -quit;
    echo "$(date +'%T') Killing rhythmbox done";
  }
}

stopsleep() {
  sleeppid=$1
  echo "$(date +'%T') stopping $sleeppid"
  set +bm
  kill $sleeppid >/dev/null 2>&1
}

runtime=${1:-1h}
mypid=$$
set -bm
rhythmbox --display :0.0 --play http://audio-ogg.ibiblio.org:8000/wcpe.ogg&
playerpid=$!
sleep $runtime& >/dev/null 2>&1
sleeppid=$!
echo "$(date +'%T') mypid=$mypid player pid=$playerpid sleeppid=$sleeppid"
trap 'stopsleep $sleeppid' CHLD INT TERM
wait $sleeppid
echo "$(date +'%T') terminating"
trap SIGCHLD
trap SIGINT
trap SIGTERM
cleanup $mypid final
wait

Note the use of set +bm in the stopsleep function to reset the job control settings and suppress the messages that you saw earlier with runclock4.sh

Listing 13 shows an example crontab that will run the alarm from 6 a.m. to 7 a.m. each weekday (Monday to Friday) and from 7 a.m. for two hours each Saturday and from 8:30 a.m. for an hour and a half each Sunday.


Listing 13. Sample crontab to run your alarm clock
             
0 6 * * 1-6 /home/ian/alarmclock.sh 1h
0 7 * * 7 /home/ian/alarmclock.sh 2h
30 8 * * 0 /home/ian/alarmclock.sh 90m

Refer to our previous tip Job scheduling with cron and at to learn how to set your own crontab for your new alarm clock.

In more complex tasks, you may have several child processes. The cleanup routine shows how to use the ps command to find the children of your script process. You can extend the idea to loop through an arbitrary set of children and terminate each one.

If you'd like to know more about administrative tasks in Linux, read the tutorial "LPI exam 102 prep: Administrative tasks,", or see the other Resources below. Don't forget to rate this page and let us know what other tips you'd like to see.

Resources

Learn

find tip...

Date: Sat, 14 Sep 1996 19:50:55 -0400 (EDT)
From: Bill Duncan <[email protected]>
Subject: find tip...

Hi Jim Murphy,

Saw your "find" tip in issue #9, and thought you might like a quicker method. I don't know about other distributions, but Slackware and Redhat come with the GNU versions of locate(1) and updatedb(1) which use an index to find the files you want. The updatedb(1) program should be run once a night from the crontab facility. To ignore certain sub-directories (like your /cdrom) use the following syntax for the crontab file:

41 5 * * *  updatedb --prunepaths="/tmp /var /proc /cdrom" > /dev/null 2>&1

This would run every morning at 5:41am, and update the database with filenames from everywhere but the subdirectories (and those below) the ones listed.

#3959 crontab administration usage and troubleshooting techniques

Common Problems/Questions and Solutions/Answers:

Q: I edited the crontab file but the commands still don't get executed.

A: Be sure user is not editing the crontab file directly with a simple text editor such as vi. Use crontab -e which will invoke
the vi editor and then signal cron that changes have been made. Cron will only read the crontab file when the daemon
is started, so if crontab has been edited directly, cron will need to be killed, /etc/cron.d/FIFO removed, and the cron daemon
restarted in order to recover the situation.

'''Q: I deleted all my crontab entries using crontab -e but crontab -l shows that they are still there.'''

A: Use crontab -r to remove an entire crontab file. Crontab -e does not know what to do with empty files,
so it does not update any changes.

Q: Can I use my **** editor ?

A: Yes, by setting the environment variable EDITOR to ****.

Q: Why do I receive email when my cron job dies?
A: Because there is no standard output for it to write to. To avoid this, redirect the output of the command to a
device (/dev/console, /dev/null) or a file.

Q: If I have a job that is running and my system goes down, will that job complete once the system is brought back up?

A: No, the job will not run again or pick up where it left off.

Q: If a job is scheduled to run at a time when the system is down, will that job run once the system is brought back up?

A: No, the job will not be executed once the system comes back up.

Q: How can I check if my cron is running correctly ?

A: Add the entry * * * * * date > /dev/console to your crontab file. It should print the date in the console every minute.

Q: How can I regulate who can use the cron.
A: The file /var/spool/cron/cron.allow can be used to regulate who can submit cron jobs.

If /var/spool/cron/cron.allow does not exist, then crontab checks /var/spool/cron/cron.deny to see who should not be
allowed to submit jobs.

If both files are missing only root can run cron jobs.

TROUBLESHOOTING CRON

If a user is experiencing a problem with cron, ask the user the following few questions to help debug the problem.

1. Is the cron daemon running?

#ps -ef |grep cron

2. Is there any cron.allow/deny file?

#ls -lt /etc/cron*

3. Is it the root crontab or a non-su crontab?

#crontab -e "USER NAME"

4. If you are calling a script through crontab, does the script run from the command line?

    	Run the script at the command line and look for errors

5. Check that the first 5 fields of an entry are VALID or NOT commented out.

(minute, hours, day of the month, month and weekday)

6. Check for crontab related patches.

(check with sunsolve and the solaris version installed on the system
 for exact patch match)

7. Check for recommended and security related patches?

(recommend to the customer to install all recommended and security patches
   relevant to the OS installed)

8. How did you edit crontab?

#crontab -e "user name"

9. How did you stop/kill the cron daemon?

#/etc/init.d/cron stop and start                      

crontab_header

Many times admins forget the field order of the crontab file
and alway reference the man pages over-and-over.

Make your life easy. Just put the field definitions in your crontab file
and comment (#) the lines out so the crontab file ignores it.

# minute (0-59),
# |      hour (0-23),
# |      |       day of the month (1-31),
# |      |       |       month of the year (1-12),
# |      |       |       |       day of the week (0-6 with 0=Sunday).
# |      |       |       |       |       commands
  3      2       *       *       0,6     /some/command/to/run
  3      2       *       *       1-5     /another/command/to/run

Sun_Solaris_AuditGuide

if both cron.allow and cron.deny files exist the cron.deny is ignored.

This can be accomplished by either listing users permitted to use the command in the file /var/spool/cron/cron.allow and the /var/spool/cron/at.allow or in the list of user not permitted to access the command in the file /var/spool/cron/cron.deny.

Dru Lavigne 09/27/2000

Recommended Links

LJ Take Command cron: Job Scheduler by Michael S. Keller

Have you ever wandered near your Linux box in the middle of the night, only to discover the hard disk working furiously? If you have, or just want a way for some task to occur at regular intervals, cron is the answer.

Debian GNU-Linux -- cron

Debian GNU-Linux -- anacron a cron-like program that does not assume that the system is running continuously.

Anacron (like `anac(h)ronistic') is a periodic command scheduler. It executes commands at intervals specified in days. Unlike cron, it does not assume that the system is running continuously. It can therefore be used to control the execution of daily, weekly and monthly jobs (or anything with a period of n days), on systems that don't run 24 hours a day. When installed and configured properly, Anacron will make sure that the commands are run at the specified intervals as closely as machine-uptime permits.

This package is pre-configured to execute the daily jobs of the Debian system. You should install this program if your system isn't powered on 24 hours a day to make sure the maintenance jobs of other Debian packages are executed each day.

CSSA-1999-023.0.txt

Reference

crontab command creates, lists or edits the file containing control statements to be interpreted by the cron command (cron table). Each statement consists of a time pattern and a command. The cron program reads your crontab file and executes the commands at the time specified in the time patterns. The commands are usually executed by a Bourne shell (sh).

The crontab command reads a file or the standard input to a directory that contains all users' crontab files. You can use crontab to remove your crontab file or display it. You cannot access other users' crontab files in the crontab directory.

COMMAND FORMAT

Following is the general format of the crontab command.

     crontab    [ file ]
     crontab -e [ username ]
     crontab -l [ username ]
     crontab -r [ username ]

Options

The following options may be used to control how crontab functions.

-e Edit your crontab file using the editor defined by the EDITOR variable.
-r Removes your current crontab file. If username is specified then remove that user's crontab file. Only root can remove other users' crontab files.
-l List the contents of your current crontab file.

Crontab File Format

The crontab file contains lines that consist of six fields separated by blanks (tabs or spaces). The first five fields are integers that specify the time the command is to be executed by cron. The following table defines the ranges and meanings of the first five fields.


Field Range Meaning

1 0-59 Minutes
2 0-23 Hours (Midnight is 0, 11 P.M. is 23)
3 1-31 Day of Month
4 1-12 Month of the Year
5 0-6 Day of the Week (Sunday is 0, Saturday is 6)

Each field can contain an integer, a range, a list, or an asterisk (*). The integers specify exact times. The ranges specify a range of times. A list consists of integers and ranges. The asterisk (*) indicates all legal values (all possible times).

The following examples illustrate the format of typical crontab time patterns.


Time Pattern Description

0 0 * * 5 Run the command only on Thursday at midnight.
0 6 1,15 * 1 Run the command at 6 a.m. on the first and fifteenth of each month and every Monday.
00,30 7-20 * * * Run the command every 30 minutes from 7 a.m. to 8 p.m. every day.


NOTE:
The day of the week and day of the month fields are interpreted separately if both are defined. To specify days to run by only one field, the other field must be set to an asterisk (*). In this case the asterisk means that no times are specified.


The sixth field contains the command that is executed by cron at the specified times. The command string is terminated by a new-line or a percent sign (%). Any text following the percent sign is sent to the command as standard input. The percent sign can be escaped by preceding it with a backslash (\%).

A line beginning with a # sign is a comment.

Each command in a crontab file is executed via the shell. The shell is invoked from your HOME directory (defined by $HOME variable). If you wish to have your (dot) .profile executed, you must specify so in the crontab file. For example,

     0 0 * * 1   . ./.profile ; databaseclnup

would cause the shell started by cron to execute your .profile, then execute the program databaseclnup. If you do not have your own .profile executed to set up your environment, cron supplies a default environment. Your HOME, LOGNAME, SHELL, and PATH variables are set. The HOME and LOGNAME are set appropriately for your login. SHELL is set to /bin/sh and PATH is set to :/bin:/usr/bin:/usr/lbin.


NOTE:
Remember not to have any read commands in your .profile which prompt for input. This causes problems when the cron job executes.


Command Output

If you do not redirect the standard output and standard error of a command executed from your crontab file, the output is mailed to you.

Access

To use the crontab command you must have access permission. Your system administrator can make the crontab command available to all users, specific users, or no users. Two files are used to control who can and cannot access the command. The cron.allow file contains a list of all users who are allowed to use crontab. The cron.deny file contains a list of all users who are denied access to crontab. If the cron.allow file exists but is empty, then all users can use the crontab command. If neither file exists, then no users other than the super-user can use crontab.

Displaying your crontab

If you have a crontab file in the system crontab area, you can list it by typing crontab -l. If you do not have a crontab file, crontab returns the following message:

crontab: can't open your crontab file.

DIAGNOSTICS AND BUGS

The crontab command will complain about various syntax errors and time patterns not being in the valid range.


CAUTION:
If you type crontab and press Return without a filename, the standard input is read as the new crontab entries. Therefore, if you inadvertently enter crontab this way and you want to exit without destroying the contents of your current crontab file, press the Del key. Do not press the Ctrl-D key; if you do, your crontab file will only contain what you have currently typed.


Related Commands

/usr/sbin/cron.d The main directory for the cron process.
/usr/sbin/cron.d/log Accounting information for cron processing.
/usr/sbin/cron.d/crontab.allow A file containing a list of users allowed to use crontab.
/usr/sbin/cron.d/crontab.deny A file containing a list of users not allowed to use crontab.
/usr/spool/cron/crontabs Location of crontab text to be executed.


cron.allow and cron.deny

Crontab supports two files:

/etc/cron.allow
/etc/cron.deny

If cron.allow exists then you MUST be listed in it to use crontab (so make sure all the system accounts like root are listed), this is very effective for limiting cron to a small number of users. If cron allow does not exists, then cron.deny is checked and if it exists then you will not be allowed to use crontab unless you are listed ("locked out")

In both cases users are listed one per line, so you can use something like:

cat /etc/passwd | cut -d":" -f 1 | fgrep -v `cat cron.deny` > /etc/cron.allow

To populate it and then delete all system accounts and unnnessary user accounts.

allow users crontab access

I assume you are on a Linux system. Then, you have a small syntax error in viewing other users crontabs, try "crontab -l -u username" instead.

Here is how it works: Two config files, /etc/cron.deny and /etc/cron.allow (on SuSE systems these files are /var/spool/cron.deny and .../allow), specify who can use crontab.

If the allow file exists, then it contains a list of all users that may submit crontabs, one per line. No unlisted user can invoke the crontab command. If the allow file does not exist, then the deny file is checked.

If neither the allow file nor the deny file exists, only root can submit crontabs.

This seems to be your case, so you should create one of these files ... on my system I have a deny file just containing user "guest", so all others are allowed.

One caveat: this access control is implemented by crontab, not by cron. If a user manages to put a crontab file into the appropriate directory by other means, cron will blindly execute ...

[from the book "Linux Administration Handbook" by Nemeth/Snyder/Hein and validated locally here]

docs.sun.com System Administration Guide Advanced Administration Controlling Access to the crontab.

You can control access to the crontab command by using two files in the /etc/cron.d directory: cron.deny and cron.allow. These files permit only specified users to perform the crontab command tasks such as creating, editing, displaying, or removing their own crontab files.

The cron.deny and cron.allow files consist of a list of user names, one per line. These access control files work together as follows:

Superuser privileges are required to edit or create the cron.deny and cron.allow files.

The cron.deny file, created during SunOS software installation, contains the following user names:
daemon
bin
smtp
nuucp
listen
nobody
noaccess

None of the user names in the default cron.deny file can access the crontab command. You can edit this file to add other user names that will be denied access to the crontab command.

No default cron.allow file is supplied. So, after Solaris software installation, all users (except the ones listed in the default cron.deny file) can access the crontab command. If you create a cron.allow file, only these users can access the crontab command.

To verify if a specific user can access crontab, use the crontab -l command while you are logged into the user account. $ crontab -l

If the user can access crontab, and already has created a crontab file, the file is displayed. Otherwise, if the user can access crontab but no crontab file exists, a message such as the following is displayed: crontab: can't open your crontab file

This user either is listed in cron.allow (if the file exists), or the user is not listed in cron.deny.

If the user cannot access the crontab command, the following message is displayed whether or not a previous crontab file exists: crontab: you are not authorized to use cron. Sorry.

This message means that either the user is not listed in cron.allow (if the file exists), or the user is listed in cron.deny.

Determining if you have crontab access is relatively easy. A Unix system administrator has two possible files to help manage the use of crontab. The administrator can explicitly give permission to specific users by entering their user identification in the file:

/etc/cron.d/cron.allow

Alternatively, the administrator can let anyone use crontab and exclude specific user with the file:

/etc/cron.d/cron.deny

To determine how your system is configured, first enter the following at the command line:

more /etc/cron.d/cron.allow

If you get the message, "/etc/cron.d/cron.allow: No such file or directory" you're probably in fat city. One last step, make sure you are not specifically excluded. Go back to the command line and enter:

more /etc/cron.d/cron.deny

If the file exists and you're not included therein, skip to setup instruction. If there are entries in the cron.allow file, and you're not among the chosen few, or if you are listed in the cron.deny file, you will have to contact the administrator and tell him/her you are an upstanding citizen and would like to be able to schedule crontab jobs.

In summary, users are permitted to use crontab if their names appear in the file /etc/cron.d/cron.allow. If that file does not exist, the file /etc/cron.d/cron.deny is checked to determine if the user should be denied access to crontab. If neither file exists, only the system administrator -- or someone with root access -- is allowed to submit a job. If cron.allow does not exist and cron.deny exists but is empty, global usage is permitted. The allow/deny files consist of one user name per line.



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2021 by Softpanorama Society. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to to buy a cup of coffee for authors of this site

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: August 10, 2020