Log rotation in RHEL/CENTOS

News	Perl Log Analysis and Management Tools	Recommended Links	Log analyzers	Log Colorizing
Perl-Logrotate	Log rotation in Solaris		Humor	Etc

Most large nad well-written Unix programs produce activity and error logs. These logs are important sources of information when debugging system problems, investigating security incidents, and monitoring system workload.

But log files grow and never shrink, and they can eventually consume all of the free space in a filesystem and interfere with the normal operation of the system.

The logrotate script helps balance record-keeping with storage limitations by periodically saving log files and starting a new ones. A pre-determined number of log files (of each type) are kept on disk, and older logs are deleted automatically.

RHEL included logrotate with the distribution and it is preconfigured for basic log-rotation. If you have some logs under /var/log that are suffixed with a dot and a number (/var/log/daemon.log.1) or even compressed (/var/log/mysql.log.5.gz), it means they were rotated by logrotate.

Logrotate needs a simple configuration file to help it manage the logs. You can have multiple configuration files, one for every log, or maintain multiple logs through one configuration file. The default configuration file is /etc/logrotate.conf. If you have services that keep a log under /var/log like Apache and MySQL, you'll find their configuration files under /etc/logrotate.d.

Here is a sample logrotate.conf:

weekly rotate 4 create include /etc/logrotate.d /var/log/wtmp { missingok monthly create 0664 root utmp rotate 1 }

The first four keywords, weekly, rotate 4, create and include are global directives, whereas the directives missingok, monthly, create and rotate are specific to the log file /var/log/wtmp. A local directive takes precedence over a global directive if it appears in both, like rotate in the listing above.

Working with Logrotate

If you have a program that keeps a log, you can save some serious effort by using logrotate to keep them in order for you. Logrotate can also work on logs of services that are run by a normal user. It also comes in handy if you have a program that keeps multiple logs and writes a lot to them.

Let's assume we have a new service "foobar" which is configured to log under /home/bodhi/logs/foobar and runs as a normal user's process. We keep its logrotate configurations under /home/bodhi/config in a file named foobar. Like this one:

/var/log/foobar/*.log { daily missingok rotate 7 compress delaycompress create 640 bodhi bodhi mail bodhi@localhost sharedscripts postrotate /etc/init.d/foobar restart endscript }

We use daily to ensure that the logs are rotated everyday and rotate 7 to make sure no more than 7 days of logs are kept. When a log expires, it's emailed to bodhi@localhost before being deleted as specified by mail. compress shrinks rotated logs except yesterday's because of delaycompress. Logrotate doesn't complain if the log files are missing thanks to missingok. After the logs have been rotated, a new one is created by create as being owned by the user and group bodhi and the service is restarted.

Putting it to work

Now that we have our configuration file in place, it's time to create a cron job for it to actually make logrotate work. You do this by running crontab -e and adding a new cron job to the file.

00 00 * * * /usr/sbin/logrotate -s /home/bodhi/config/logrotate.status /home/bodhi/config/foobar

Cron ensures that the command runs at midnight everyday. The command has three parts. /usr/sbin/logrotate is the path to logrotate. The -s /home/bodhi/config/logrotate.status option specifies where logrotate keeps its status information. This file has to be writeable by the user running the cron. The last argument /home/bodhi/config/foobar tells logrotate to use our custom configuration file.

To test the configuration file, issue the command specified in the cron job with an additional -f switch. This will force the logs to be rotated. It helps to run logrotate with this switch every time the configuration files are changed.

Logrotate is a very simple but extremely useful tool. Many people fail to realize that logrotate can work for them as well. In this article, the configuration file instructs logrotate to take charge of a process running under a user.

We used several configuration options, but a visit to man logrotate for more options is a good idea before writing your own. If you have logrotate managing logs on your machine, reviewing its configuration files of the services its maintaining is a good way to learn logrotate's common usage.

Old News

Changing Apache log rotation behaviour on CentOS / RHEL

March 22, 2008

On a default install of CentOS or Red Hat Enterprise Linux, the log rotation script will automatically rotate the Apache log file each day and then reload the httpd service. This post looks at how to prevent this action from occuring automatically, or to change the behaviour to rotate the log files if your naming convention for log files is different from the default.

The cron daemon on a CentOS or Red Hat Enterprise Linux server by default runs the scripts in the directory /etc/cron.daily on a daily basis. This includes running the "logrotate" script which runs /usr/sbin/logrotate passing it the /etc/logrotate.conf configuration file. The default version of this logrotate.conf file is as follows:
# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4



# create new (empty) log files after rotating old ones
create


# uncomment this if you want your log files compressed
#compress


# RPM packages drop log rotation information into this directory
include /etc/logrotate.d


# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    minsize 1M
    create 0664 root utmp
    rotate 1
}


# system-specific logs may be also be configured here.
These default variables are fairly straight forward and give you control over how frequently the log files are rotated, how many lots of previous log files to keep before deleting them, whether or not to compress them and so on. It also tells logrotate to process all the files in the directory /etc/logrotate.d

The file which controls log rotation of Apache on CentOS and Red Hat Enterprise Linux is named /etc/logrotate.d/httpd, and the default contents of this file are as follows:
/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}
What this file is saying, is to rotate all files that match the pattern /var/log/httpd/*.log. The default access and error log files in Apache on CentOS and RHEL are named "access_log" and "error_log" so they match this pattern.

If you have changed the default names of the log files and they don't match this pattern, then you can have logrotate rotate your log files by making sure the pattern is in this file in place of the /var/log/httpd/*.log line. For example, if your log files were named error.foo and access.foo then /var/log/httpd/*.foo would match them.

If you do not wish the logrotate script to rotate your Apache log files at all, then you can simply delete this file like so, logged in either as root or using sudo:
rm -f /etc/logrotate.d/httpd
The changes will take effect the next time the logrotate script is run.