A case study in scripting for system audit compliance
30 Apr 2007
Think you have a secure Linux® system? Following best practices during installation
and setup is a must, but if you haven't set up regular system auditing,
you're missing half the picture. This article discusses some existing tools
and offers a couple of sample scripts to automate the process in a real-world
environment.
Many articles and books have been written on how to install a secure Linux
system. But once the system has been installed to meet security requirements,
only half the battle is won. The second half involves ensuring that the system
continues to meet its security requirements throughout its lifetime (and that
you can prove it). This means that periodic system auditing is required to make
sure that nothing goes wrong.
System audit requirements
The security requirements that you verify during routine system auditing
should be the same requirements and security principles that guide the system
installation. The three-part developerWorks series "Securing
Linux" gives you an introduction on how to install a reasonably secure Linux
system. Regular system auditing will also help refine the security policy used
for new machine installations as it helps close the feedback loop on what subsystems
are actually in use.
The first tools for meeting these requirements are system auditing
and host-based intrusion detection. This article focuses on system auditing.
Host-based intrusion detection systems such as tripwire, AIDE, and Samhain detect
when changes have been made to the file system and are therefore critical tools
for ensuring that the system retains its known state. The Linux Gazette
has an interesting article, "Constructive Paranoia," on using these tools (see
Resources section below for a link).
This article focuses specifically on the practical aspects of periodic system
auditing based on real-world requirements from a system administrator of a subnet
in a large academic network. The lessons learned by this administrator apply
to everyone from business intranets to home users who want to prevent their
home machine from becoming a zombie in the bot army. The administrator's system
is required to undergo periodic, random system audits, during which routine
audit activities (such as showing that the audit and system logs are regularly
reviewed, and checking for user accounts that have lapsed), In addition, the
administrator also has to address the following:
- Justification for suid/sgid executables that are on the system and why
they are suid/sgid
- Proof that no file system with a world writable directory (/tmp and
/var/tmp) has any suid/sgid files
- Open ports and the impact of firewalling off of those ports
What are all those suid files for?
Identifying the suid and sgid files on your system — and disabling the unnecessary
ones — is one of the fundamental rules of installing a secure system. This task
is so common that the man page for find lists the parameters for
this in its examples. Listing 1 is a script that executes the typical
find command and also helps answer the question of what the suid file
does and what package it belongs to, in order to help the administrator to identify
it, and decide if it should stay on the system or be removed. (You can download
the code for Listings 1, 3, and 4 from the zip file in the
Download section later in this article.)
Listing 1. Abbreviated sample output on suid/sgid files
[root@localhost hpc]# ./find_setuids.pl /
04755 root /usr/X11R6/bin/cardinfo
cardinfo - PCMCIA card monitor and control utility for X
pcmcia-cardinfo-3.2.7-107.3
04755 root /usr/bin/opiepasswd
opiepasswd - Change or set a user's password for the
OPIE authentication system.
Opie-2.4-544.1
04755 root /usr/bin/opiesu
opiesu - Replacement su(1) program that uses OPIE challenges
opie-2.4-544.1
04755 root /usr/bin/sudo
sudo - execute a command as another user
sudo-1.6.7p5-117.4
|
Which file systems contain world
writable directories?
As an administrator, you might be interested in world writable directories
to meet the requirement that all user-writable file systems be mounted with
the nosuid attribute. User-writable directories include users' home directories
as well as any world writable directories. This requirement is in place to prevent
users from creating suid executables that another user or administrator might
inadvertently execute. However, if a legitimate suid executable is on the same
file system as a world writable directory and is thus mounted nosuid, the suid
bits are ignored and the executable will not operate correctly. You might consider
implementing this restriction on your multi-user systems as well.
The script in Listing 1 also tests each regular file system for world writable
directories and reports whether the file system contains a world writable directory
at the end of the output. For each suid/sgid file, it also reports whether the
file is on a filesystem that contains world writable directories.
Abbreviated example output on world writable directories:
/ Contains both suid/sgid files and world writable directories.
What network ports does the system
use and for what purpose?
There are several ways to detect which ports are in use on your system. Nmap,
netstat, and lsof are the most helpful tools.
- Nmap is an extremely flexible tool that can do active and passive
scanning of remote (and local) systems.
- Netstat shows the network information about the local system.
By default, it shows open connections.
- Lsof lists open files on the system. It can be used to get information
about port usage because it also shows information about network sockets.
Kurt Seifried maintains a listing of 8,457 commonly used ports. (A link to
his ports list is in the Resources below.) You can
use this data to help explain what the port is being used for and what the impact
would be of firewalling it off. He includes information about ports that are
commonly used for trojans and root kits; for example, 31337 is commonly used
by Back Orifice and 12345, 12346, and 20034 are used by netbus.
Listing 2 contains a script that uses lsof and netstat to show the system's
current port usage in an easily readable format.
Listing 2. Sample output from port_scan.sh:
[root@localhost hpc]# ./port_scan.sh
please wait...
PORT SERVICE LINK
22 sshd
http://www.seifried.org/security/ports/0/22.html
25 sendmail
http://www.seifried.org/security/ports/0/25.html
123 ntpd
http://www.seifried.org/security/ports/0/123.html
631 cupsd
http://www.seifried.org/security/ports/0/631.html
46336 <-> 22 ssh **
|
In Listing 2, the low-number ports (<1024) indicate daemons running on the system
that accept incoming communication unless firewalled off. The high-number port
46336 shows an outgoing ssh connection and the port (22) that it is connected
to on the other end. This means that blocking outbound communication on ephemeral
ports will break commonly used client programs such as ssh. See Kurt's ports
list in Resources for more details on the effects of
firewalling the higher-number ports.These scripts and tools show port usage
at a point in time. The audit subsystem can be used to find out which ports
have been used (for the duration of the audit log files) even if they are not
currently in use. Adding the following audit rule to /etc/audit.rules will log
calls to bind.
-a entry,always -S socketcall -F a0=2
The parameter -a entry,always indicates that the rule should
always be invoked at the beginning of the system call execution. The -S
socketcall indicates that this audit rule is for the socketcall syscall.
The socketcall syscall is multiplexed on the i386 architecture, so the
-F a0=2 is required to limit the audit records generated to bind only.
Other architectures handle the bind system call differently, so these commands
and scripts will have to be altered slightly to handle architectures other than
i386. Audit events are recorded as multiple audit records that are correlated
by a shared serial number. ausearch will correlate the related records using
the serial number and present them as a group. The -i flag requests that numeric
values, indicating, for example, that saddr (IP address) and uid (user name)
be translated to human readable text when possible.
Listing 3. Abbreviated output from ausearch
# ausearch -i -sc socketcall
Abbreviated example output
----
type=SOCKETCALL msg=audit(11/20/2006 11:28:43.844:10) : nargs=3 a0=0
a1=b8004004 a2=10
type=SOCKADDR msg=audit(11/20/2006 11:28:43.844:10) : saddr=inet
host:127.0.0.1 serv:631
type=SYSCALL msg=audit(11/20/2006 11:28:43.844:10) : arch=i386
syscall=socketcall(bind) success=yes exit=0 a0=2 a1=bfffaca0 a2=b8000664
a3=1 items=0 pid=3340 auid=unknown(4294967295) uid=root gid=root euid=root
suid=root fsuid=root egid=root sgid=root fsgid=root comm=cupsd
exe=/usr/sbin/cupsd
----
type=SOCKETCALL msg=audit(11/20/2006 16:40:46.169:16) : nargs=3 a0=6
a1=b8056720 a2=10
type=SOCKADDR msg=audit(11/20/2006 16:40:46.169:16) : saddr=inet
host:192.0.34.166 serv:123
type=SYSCALL msg=audit(11/20/2006 16:40:46.169:16) : arch=i386
syscall=socketcall(bind) success=yes exit=0 a0=2 a1=bffff9a0 a2=b80004a8
a3=6 items=0 pid=3523 auid=unknown(4294967295) uid=root gid=root euid=root
suid=root fsuid=root egid=root sgid=root fsgid=root comm=ntpd
exe=/usr/sbin/ntpd
|
This output shows that each call to bind generated three audit records.
The first record type is the SOCKETCALL record, which shows the number and value
of the arguments passed to bind on entry. The second record type is the SOCKADDR
record, which shows the host by IP address and the port used. The third record
type is the SYSCALL record, which shows whether the call to bind was successful,
the arguments upon exit, gives the process ID, file executed, and shows the
effective and real user and group information of the process that made the call.
For our purposes, we are interested mostly in the serv: part of
the SOCKADDR record, which documents the port used and the exe=
field of the SYSCALL record, which documents the program that made the call
to bind.
Listing 4 contains a simple script using sed and awk to compress the output
of ausearch down to only the non-duplicated (time omitted) executable and port
number fields.
Listing 4. Sample output from auportprint.sh:
[root@localhost hpc]# ./auportprint.sh
22 /usr/sbin/sshd
123 /usr/sbin/ntpd
123 /usr/sbin/ntpdate
631 /usr/sbin/cupsd
|
This article introduced some tools and techniques that can help you maintain
your system's adherence to its security policy. It also gave you some simple
scripts to help parse system data into an easily readable format that can help
you prove the security status of the system quickly and easily to anyone less
familiar with the tools and system. I hope that you can use these simple tools
and techniques to create your own scripts to periodically test the security
stance of your systems.
Acknowledgments
Many thanks to Kylene Hall, Dustin Kirkland, Flavio Ivan da Silva, Rodrigo
Rubira Branco, and Flavio C. Buccianti for their code, suggestions, and hard
work on this article.
Download
| Description |
Name |
Size |
Download method |
| Code for Listings 1, 3, and 4 in this
article |
hpc.zip |
4KB |
HTTP |
Resources
Learn
Get products and technologies
-
Order the SEK for Linux, a two-DVD set containing the latest IBM trial
software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
- With
IBM trial software, available for download directly from developerWorks,
build your next development project on Linux.
Discuss
About the author
Emily Ratliff works in the IBM Linux Technology Center on Linux security.
She has worked on multiple Common Criteria evaluations, Trusted Computing, Kerberos,
and Enterprise Identity Mapping, among other projects. Emily's ultimate goal
is to protect her unattended workstation from her toddler's random (and surprisingly
successful) attacks.
This month we continue our look at the steps required to
harden a Linux system by considering how Linux services can be secured. In fact,
securing system services -- subsystems like printing and electronic mail, file
and web serving, and remote access (telnet, ftp, rlogin, rsh, imap, and
so on) -- represents a large part of the hardening task.
In general, the guiding principle of securing system services is install
only those packages that the system actually needs. As we noted last month,
potential intruders can't exploit software that isn't present on the system.
When removing a package isn't possible or practical, then unneeded services
should be disabled.
The following list represents the general tasks associated with securing
system services:
- Disable or remove all unneeded services.
- Specify logging and access control for all services, allowing only the
minimum access necessary.
- When appropriate, use chroot to run a service in a confined directory.
- If at all possible, create a special user to run server processes. For
example, run named as user named.
- Use secure versions of daemons when they are available.
- Restrict access to facilities services like cron to system administrators.
- If a server lets you limit the number of daemon processes, specify a
limit. Setting limits can help prevent some denial of service attacks.
- Finally, be sure to secure all services, whether they seem security-related
or not (e.g., printing).
Locating and Disabling Linux Services
Before you disable unnecessary services -- or even decide which ones to keep
-- you have to find them all. System daemons are started mostly during the boot
process and are started from many different locations, including the following:
- From /etc/inittab: some daemons are started directly by init
at boot time.
- By one of the boot scripts in /etc/init.d: these scripts are
run at boot time when they are linked to the appropriate /etc/rcN.d
subdirectory, where N indicates the run level.
- By the super network daemon, inetd or xinetd: some daemons
are started on demand, when a client requests their services.
- Periodically, using the cron scheduler: daemons can be started
via crontab entries.
- Manually: the system administrator can also start daemons and whole
subsystems by entering commands by hand.
The method required to disable a service depends on how the service was started.
Let's look at each of the possibilities individually.
Reviewing /etc/inittab
/etc/inittab file is the configuration file used by the init
daemon. At boot time, init reads the entries in the file and starts a
process according to each description. The same process occurs whenever the
system changes run levels.
On Linux systems, it is somewhat unusual for system daemons to be started
from this file, but they can be. For example, the following inittab entry
runs the script /etc/start_acct, which starts the process accounting
subsystem:
acct:35:once:/etc/start_acct
The fields in the inittab entry are (in order) the entry's label,
the run levels to which it applies, a mode keyword, and the command to execute,
along with any arguments.
You should understand the purpose and effect of all of the inittab
entries on your system. To disable a service started via inittab, simply
delete the line or comment it out by placing a number sign (#) at the
beginning of the line.
Boot Scripts
Most system services are started by a system boot script located
in /etc/init.d. Generally, each script is limited to controlling
(starting or stopping) a single service or subsystem.
To make one of these scripts run during the boot process, the script
must be linked to one of the directories in /etc/rc.d named
rcN.d, where N is the run level number. For example,
/etc/rc.d/rc3.d contains the scripts that run when the system
enters run level 3. The links in these run level-specific subdirectories
have names of the form XMMname, where X is either
K or S, MM is a two digit number, that determines
the order in which scripts are run, and name is generally the
name of the script in /etc/init.d to which this file is linked.
K-files are used to stop services when entering a run level; S-files
are used to start them (the scripts are passed the parameters stop
and start, respectively).
Here is a small portion of the directory listing from the directory
/etc/rc.d/rc5.d illustrating these links:
$ cd /etc/rc.d/rc5.d; ls -l ??[0-9]r*
lrwxrwxrwx K46radvd -> ../init.d/radvd*
lrwxrwxrwx K55routed -> ../init.d/routed*
lrwxrwxrwx S20random -> ../init.d/random*
lrwxrwxrwx S97rhnsd -> ../init.d/rhnsd*
This listing shows links to the files in /etc/init.d whose
names start with "r." There are two K-files and two S-files.
If you want to disable a service that is started via a boot script,
then you have two choices:
- You can remove all of the links to the corresponding file(s)
in /etc/init.d. If you choose this method, then be sure to
identify all of the files that are used for that service (some services
like NFS use more than one script), and then remove the associated
links in every rcN.d subdirectory.
- You can replace the existing script with a dummy version. This
method has the advantage that you can easily reactivate the service
later if necessary. For example, to disable the LDAP service, you
could use the following commands:
# cd /etc/init.d
# ./ldap stop
# mv ldap ldap.save
# echo > ldap
#!/bin/bash
exit 0
^D
# chmod 755 ldap
First, we stop the LDAP servers if they are currently running. The
remaining commands save the existing script under another name and replace
it with a dummy script.
Configuring inetd and xinetd
There is another class of network services present on Linux systems:
ones that are invoked on demand when a client requests their services.
The daemons associated with these services are controlled by the
inetd (or xinetd) "super" daemon.
inetd is responsible for starting daemons when their services
are needed (inetd itself is started at boot time by a boot script
in /etc/init.d). The list of daemons managed by inetd
include many common user-level network services, including telnet,
ftp, remote login and shells, mail retrieval, and the like.
Traditionally, inetd is configured via the file /etc/inetd.conf.
Entries for most common services are typically included by default in
the version of this file installed with the operating system. Some add-on
services also add entries to the file (e.g., Samba). Occasionally, you
may need to add entries manually for a new service that you install.
Disabling entries in /etc/inetd.conf is very easy: simply
add a number sign (#) to the beginning of the line corresponding
to the daemon you want to disable. Whenever you modify this file, you
must send the HUP signal to the inetd process in order
for it to take effect:
# kill -HUP `cat /var/run/inetd.pid`
Recent Red Hat Linux versions and many other distributions have switched
from the standard inetd facility to a new one called xinetd.
xinetd performs a similar function, but is configured differently.
Each subordinate daemon has a separate configuration file in /etc/xinetd.d,
and each file is named for the daemon it corresponds to. Here is a simple
example file, /etc/xinetd.d/rlogin:
service rlogin
{
socket_type = stream
protocol = tcp
wait = no
user = root
server = /usr/sbin/in.rlogind
server_args = -l
disable = yes
}
The various attributes correspond to the fields in traditional
/etc/inetd.conf entries. The final attribute, disable,
controls whether the daemon is active or not. In this case, the rlogin
service is disabled.
Obviously, disabling a subdaemon under xinetd's control is
done by setting the disable attribute to yes in the
appropriate configuration file.
Disabling or Removing Services
You should examine the systems that you administer carefully and
be sure that you know what's running on them. Unfortunately, Linux distributions
often install and enable a variety of unneeded services. Table One
includes services that you might consider disabling or removing when
they're not needed. Note that this list is not comprehensive and that
there is no substitute for examining your own systems.
Table One: Services
you may not need
IN /ETC/INITTAB:
- Comment out or remove entries for getty processes
for terminal lines which will not be used.
DAEMONS STARTED BY BOOT SCRIPTS (WITH TYPICAL SCRIPT
NAMES):
- NFS automounter (and NFS itself if it's not used):
autofs, nfs, nfsserver, nfslock.
- Unused standard networking servers, for example:
routed, gated, ratvd, snmpd,
named, dhcpd, dhclient, dhrelay,
nscd, smb.
- Unneeded mail transport services: sendmail,
postfix.
- Unused optional local and network facilities:
atd, ldap, kudzu, rhnsd, vncserver,
ypbind, quota, quotad, apache,
mysql, hylafax, netsaint (and dozens
more).
- Unneeded printing facililites: lpr, lprng,
cups.
SUBDAEMONS OF INETD/XINETD:
- User remote access: rlogind, rshd,
telnetd, ftpd, tftpd, rexecd,
rexd, talkd, ntalkd, fingerd,
and so on.
- TCP/IP connectivity testing services: echo, daytime,
chargen, discard, time.
- Unused mail retrieval protocols: imap,pop2,
pop3, etc.
|
Note that many recent Linux systems disable all inetd/xinetd
subdaemons by default. At present, this is considered a "best practice".
Restricting Access to Services
Once you've removed or disabled unneeded services, the next step
is securing the ones that will run.
ACCESS CONTROL FOR INETD AND XINETD
Let's look again at a sample entry from /etc/inetd.conf:
#service socket proto wait? user prog args
imap stream tcp nowait root /usr/sbin/tcpd/usr/sbin/imapd
On most Linux systems, inetd is set up to use Wietse Venema's
TCP Wrappers package. When TCP Wrappers is in use, the tcpd program
replaces the native program for each inetd service (the sixth
field in each configuration file entry). The actual service daemon is
given as tcpd's argument, and any arguments to the service daemon
itself follow the image name.
TCP Wrappers provides two important additions to the standard
inetd functionality: access control and logging. ItautomaticaTCP
Wrappersy logs access failure to the syslog daemon facility.
Using TCP Wrappers, the files /etc/hosts.allow and /etc/hosts.deny
control access to the various daemons under inetd's control,
according to the following logic:
- If an entry in hosts.allow grants access, then the request
is honored. The first applicable line in /etc/hosts.allow
is used.
- When no line in hosts.allow applies, hosts.deny
is checked next. If any entry within it denies access, then the
request is denied. Again, the first applicable entry is used.
- In all other cases, the request is granted.
Here are some sample entries from hosts.allow:
rlogin : ophelia hamlet laertes yorick
pop3 : .ahania.com EXCEPT mailhost.ahania.com
ftpd : 192.168.4 192.168.10
The first entry grants access to the rlogin service to users
on any of the listed hosts (hostnames may be separated by commas and
or spaces). The second entry allows email retrieval via POP3 by users
from any host in the domain ahania.com except mailhost.
The third entry allows ftp access to all hosts on the subnets
192.168.4 and 192.168.10.
Here is a simple /etc/hosts.deny file:
The sole entry in this file prevents all access that has not been
explicitly permitted in hosts.allow. Using such an entry as the
final line in this file is strongly recommended.
xinetd subdaemons are generally secured in a different way
(although TCP Wrappers may also be used in combination with xinetd).
Here is a sample entry for the imapd daemon which illustrates
xinetd access control:
service imap
{
user = root
server = /usr/sbin/imapd
only_from = 192.168.10.0 localhost
no_access = dalton.ahania.com
access_times = 07:00-20:00
}
The only_from attribute specifies the hosts that are allowed
to use this service; requests from any remote host not on the specified
subnet will be refused. The no_access attribute performs the
opposite function and denies access to the specified host(s). Finally,
the access_times attribute specifies when the service is available
to users who are allowed to use it.
xinetd also provides excellent logging facilities, which should
be used. Logging is often specified in the facility's main configuration
file, /etc/xinetd.conf, as in this example from Red Hat:
defaults
{
log_type = SYSLOG authpriv
log_on_success = HOST PID
log_on_failure = HOST
}
These entries specify that success and failure will be logged to
the syslog authpriv facility. Successes will log the
remote hostname and process ID, and failures will log the remote hostname.
You can add additional logging items for a specific service by including
an entry like this one within its individual configuration file:
This entry adds the user ID to the log entry for successful connections.
USING CHROOT AND SANDBOXING
Isolating potentially dangerous services from the rest of the system
is one way to minimize risks. The chroot command is used for
this purpose.
chroot runs a command using a specified location within the
file system as the command's root directory (rather than /).
For example, the following command runs the named daemon, using
the directory /jail as its root directory:
The named process will treat /jail as its root directory.
For example, when named looks for its main configuration file,
usually /etc/named.conf, it will actually access the file
/jail/etc/named.conf. (Note that recent versions of named
have a -t option which can be used to specify an alternate
root location directly, without needing chroot.)
In order for named to function correctly, you will need to
set up a minimal file system under /jail that contains the files
and directories that named will need: /etc/named.conf and the
zone files, the named executable and any required libraries (you can
link named statically when you build it to avoid the latter),
the password and group files, and the device files null and
random. The latter must be created with the mknod command:
# mknod /jail/dev/null c 1 3
# mknod /jail/dev/random c 1 8
Running a daemon or subsystem as a user specifically created for
that purpose (rather than root) is sometimes called sandboxing,
and it's best to use this security technique wherever you can. You can
combine sandboxing and chroot for added security.
For example, you can create a special user and group to run the
named daemon (the user and group are often also called named).
Then, use the named command's -u option to start the
server running as the specified user. Under this scheme, you will also
need to change the ownership of the named directory tree to this
user and group.
Many facilities have a built-in ability to specify a user other than
root as the user context for the daemon process. You can manually configure
ones that don't by creating a special user, changing the ownership of
important commands and files to that user, and granting that user access
to any required system resources. It may take some trial and error to
sort out all the details, but most daemons can be secured in this way.
Other Service Configuration Issues
We'll close this column with a few miscellaneous service security
issues.
Some packages provide ways of limiting access to the services as
a whole (e.g., cron) or to administrative functions (e.g., LPRng).
For example, the files cron.allow and cron.deny control
access to the crontab command. The location of these files varies:
for Red Hat, they're in /etc; for SuSE, they're in /var/spool/cron.
Both files contain a list of user names, one per line, which determine
access to crontab. If cron.allow exists, then a username
must be listed in it to run crontab. If cron.allow does
not exist but cron.deny does exist, then any user not listed
in cron.deny may use the crontab command. cron.deny
may be empty to allow unlimited access. Finally, if neither file exists,
then anyone can use crontab. Be aware that these files control
only whether a user can use the crontab command, and not whether
any existing crontab entries will be executed.
Some services provide both normal and secure daemons. When a secure
service option is available, the service should be configured to use
it.
Next month, we'll conlude our look at hardening Linux systems and
examine techniques for securing local file systems, user authentication,
remote system access, and ongoing system monitoring.
Æleen Frisch is the author of Essential System Administration.
She can be reached at aefrisch@lorentzian.com.
