Configuring Sendmail on the Solaris 9 Platform

By Eric Rinker

The goal of this article is to impart a basic understanding of how to make changes to Sendmail on a machine running the Solaris 9 Operating System. This article is written for engineers with a reasonably good working knowledge of the standard principles of the UNIX operating system. To utilize this article, you need to know how to edit files and run programs, and you need root access.

How Email Works (Mail Transfer Agents)

Two categories of application deal with email: Mail User Agents (MUAs) and Mail Transfer Agents (MTAs).

Mail User Agents are applications that facilitate the creation, viewing, and disposal of email messages. Examples include mail or elm in a UNIX environment, and Eudora or Outlook in the Windows world. Netscape and Explorer are Internet browsers that can also double as MUAs.

Mail Transfer Agents transport email from one machine; typically, each machine uses only one MTA. Sendmail fills this role, while other MTAs out there include Exim, Postfix, and Qmail.

Evolution of Sendmail

Sendmail is one of the oldest and mostly widely used MTAs in the world. It is the default MTA for most UNIX distributions, including HP's HP-UX, IBM's AIX, and Sun Microsystems' Solaris OS. Sendmail's long life has made it complicated to configure and maintain, but it makes up for its drawbacks with its ability to do just about anything.

First appearing over 30 years ago, Sendmail has evolved into a robust, feature-rich method for transporting electronic mail from one location to another. Originally designed at a time when hard drives the size of washing machines supplied 64 kilobytes of usable storage, Sendmail used every trick in the book to conserve space. To make everything short and to the point, the Sendmail configuration file used such cryptic parameters as "Fw" for "Domains we receive mail for" and "DH" for "Who gets all local email." While there is a method to the madness, it is not readily apparent to the novice user. For backwards compatibility, these cryptic parameters are still present in the configuration file of today's Sendmail versions.

Over the years, as features were added to Sendmail, the configuration process became more and more complicated. To make it more administrator-friendly, Sendmail uses a m4-based compilation and configuration model. This layer between the administrator and the build and configuration process makes Sendmail easier to set up and maintain without requiring upgrades of older programs to handle new interaction methods.

Configurations

This document couldn't possibly cover everything there is to know about Sendmail without being hundreds of pages long, and a bore to read. Instead, we focus on three commonly seen configurations: Mail Server, Incoming Relay, and Outgoing Only.

When modifying the behavior of Sendmail, the /etc/mail/sendmail.cf file is not directly altered. Instead, a .mc file is altered and run through the m4 macro processor. Some example .mc files are in /usr/lib/mail/cf:

For our examples, we will copy the main.mc file to new.mc and make our modifications like so:

cd /usr/lib/mail/cf
vi new.mc
make new.cf
cp new.cf /etc/mail/sendmail.cf
/etc/init.d/sendmail restart

To begin with, common elements are shared in all three configurations. A minimal file contains the following:

OSTYPE(`solaris8')dnl
DOMAIN(`generic')dnl
MAILER(`local')dnl
MAILER(`smtp')dnl

The OSTYPE macro defines what system this file is on.
The DOMAIN macro is used to pull in another file into the resulting sendmail.cf file.
The MAILER macros define which of the many different delivery methods this configuration file will use.
In this example, we are on a Solaris 8 or higher system, we are including the "generic" domain file, and we want to use both the local delivery system and the SMTP system.

Mail Server
 

The mail server is your typical server for incoming mail. It receives mail for user@domain, delivers it to the user's local mailbox, and processes mail in its queue for delivery to the outside world.

You only need to make one change: Add each domain that is to be considered a local account into /etc/mail/local-host-names.

End result:

OSTYPE(`solaris8')dnl
MAILER(`local')dnl
MAILER(`smtp')dnl

Incoming Relay
 

Incoming Relay is the common configuration for company email servers that are outside of the company firewall. Instead of storing the email, these relays pass it on to a predefined server inside the firewall that is the company's mail server. This setup is perfect for implementing filtering, since this machine doesn't handle the other duties of your typical mail server.

To configure Incoming Relay, we first need to add the relay server information. In this case, we are going to relay everything to relay.mydomain.com.

define(`SMART_HOST', 'relay.mydomain.com')dnl

Next, we have to allow mail to be relayed through this machine. It's best to only relay mail for domains served by the internal servers. The following option tells sendmail to use the /etc/mail/relay-domains file as a list of domains allowed to send or receive mail through this server:

FEATURE(`relay_entire_domain')dnl

We are done. This server will now relay for any domains in the /etc/mail/relay-domains file, except for local accounts, to relay.mydomain.com.

End result:

OSTYPE(`solaris8')dnl
DOMAIN(`solaris-antispam')dnl
define(`SMART_HOST', 'relay.mydomain.com')dnl
FEATURE(`relay_entire_domain')dnl
MAILER(`local')dnl
MAILER(`smtp')dnl

Outgoing Only
 

For security purposes, it's best not to set up an indiscriminate mail relay. Every machine needs to use an MTA to send email, and some programs require the ability to relay emails through an SMTP server. With these requirements, you can both relay mail for local services and secure your system from becoming an open relay by configuring Sendmail to attach only to the loop-back address.

To make Sendmail outgoing only, it needs to not accept mail from any remote hosts. To do this, we force it to use only the local loop-back address.

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')

No other options are required; Sendmail transports mail from the local machine to the outside world by default.

End result:

OSTYPE(`solaris8')dnl
DOMAIN(`solaris-generic')dnl
DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')
MAILER(`local')dnl
MAILER(`smtp')dnl

 

Where to Go Next

For more information on options for a Relay server, see the sendmail.org tip Allowing controlled SMTP relaying in Sendmail 8.9 and later. Specifically, look at using the access_db option for a more robust anti-spam filtering relay server.

Now that you know how to make changes, you can decide what kinds of changes you want to make. Your first stop should be /usr/lib/mail/README as it contains a good deal of information about Sendmail configuration, the m4 macros, and so on. Once you've exhausted that, you can check out some of the following resources.

Books:

Sites:

Acknowledgments

The author would like to thank John Beck of Sun Microsystems for his help in reviewing this article.

April 2003

Unless otherwise licensed, code in all technical manuals herein (including articles, FAQs, samples) is provided under this License.