Softpanorama

Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
May the source be with you, but remember the KISS principle ;-)
Skepticism and critical thinking is not panacea, but can help to understand the world better

SSH Forwarding Minitutorial

SSH can be used to secure connections between systems. It allows application traffic such as that generated by TELNET, FTP POP3, or even X Windows to be both encrypted and compressed. Compression is useful over slow modem links. Implementations can allow the user a choice of encryption methods. Client software often offers both SSH1 and SSH2 support. The user is authenticated by password or public/private key.

The SSH suite includes the ssh program which replaces rlogin and telnet, scp which replaces rcp, and sftp which replaces ftp. Also included is sshd which is the server side of the package, and the other basic utilities like ssh-add, ssh-agent, ssh-keysign, ssh-keyscan, ssh-keygen and sftp-server. OpenSSH supports SSH protocol versions 1.3, 1.5, and 2.0.

OpenSSH is primarily developed by the OpenBSD Project, and its first inclusion into an operating system was in OpenBSD 2.6. The software is developed outside the USA, using code from roughly 10 countries, and is freely useable and re-useable by everyone under a BSD license.

Managing the distribution of OpenSSH is split into two teams. One team does strictly OpenBSD-based development, aiming to produce code that is as clean, simple, and secure as possible. We believe that simplicity without the portability "goop" allows for better code quality control and easier review. The other team then takes the clean version and makes it portable, by adding the portability "goop" so that it will run on many operating systems (these are known as the p releases, and named like "OpenSSH 4.0p1"). Please click on the provided link for your operating system.

One major benefit of SSH, besides the obvious advantage of cryptographically secure connections, is that it allows you to log on to a server without ever having to type your password. You do have to type a password (only once -- but we'll get to that in a minute), but it doesn't have to be the password for your account on the server, and it will be the same password for every system to which you log in. That is possible through the magic of authentication via cryptographic keys.

Normally, when you log in to a system, you authenticate by entering your password for that system. Your password goes, as it is typed, to the remote system, which authenticates it against the /etc/passwd or /etc/shadow file. By contrast, SSH uses an authentication method with public-key cryptography. Essentially, it's a challenge-response mechanism: You can authenticate against a public key on the server, so when the client connects, the server encrypts a random number with the public key. If the client possesses the private key, then it can decrypt the random number and report it back to the server. That proves to the server that the person logging in has the authorized private key. The private key is further protected when you encrypt it with a password. The password you type to authenticate via public key cryptography is the password for your private key.

That sort of authentication requires you to first generate a pair of public and private keys. The ssh-keygen command does that. Because of the different versions of the SSH protocol out there, OpenSSH 2.5 and higher can generate three different types of keys: rsa1, for compatibility with SSH version 1 clients; DSA, for connecting to SSH protocol version 2 clients using the Digital Signature Algorithm; and RSA, for connecting to SSH protocol version 2 clients using the standard RSA algorithm. You can set the type of key SSH generates with the -t option. Thus to generate your rsa2 public and private key pair, you would run ssh-keygen -t rsa2. For the three key types, the keys generated are by default stored in your .ssh directory beneath your home directory. The rsa1 keys are named identity, DSA keys are named id_dsa, and protocol version 2 RSA keys are named id_rsa. Each key has a corresponding public key with a .pub extension. You want to check the permissions on the private keys and make sure that they're not world-readable; these are secrets not intended for sharing.

To allow authentication, you must make the public key available on the computer into which you would like to log in. The name of the file into which you put the public key is $HOME/.ssh/authorized_keys. Each public key is a single line, so you can use scp to copy the public keys to the destination system and then, on the remote system, type cat id_dsa.pub >> .ssh/authorized_keys. After that you can authenticate without sending any sort of password over the network. (The private key is decrypted on the client only.)

You can set your system to require authentication only once. You can do that with the SSH authentication agent, ssh-agent. The ssh-agent syntax is rather strange: because it must set some shell variables, you must use eval `ssh-agent`. The ssh-agent tries to determine the shell syntax necessary, but occasionally it fails and you need to pass it the -s option for Bourne shell derivatives such as ksh and zsh, or the -c option for C Shell derivatives.

You must execute the ssh-agent in your top-level shell for maximum usability. For a console or ssh-based session that means in your top login shell or, for X-based sessions, in your .Xclients or another startup script. In either case, you must kill the ssh-agent with ssh-agent -k prior to quitting, or lots of startup files will be left around.

The ssh-agent only manages keys; you have to tell it which keys to use. To do that, you can ssh-add private-key-file, where private-key-file is $HOME/.ssh/id_dsa or one of your other private keys. Once you type the passphrase for your key, ssh-agent remembers it and automatically handles the key requests from new SSH sessions. To handle all contingencies, I add all three of my keys at login time and forget about it until I'm ready to quit. I end up typing three passwords, and then I need not enter any more passwords for the rest of the day.

If I plan on connecting through one server to another server, I connect via SSH to that first server with the -A option, which allows the server to pass any authentication requests it gets back to the initial client. (You can make that operation the default by setting ForwardAgent yes in your ssh configuration file. The per-user configuration file is $HOME/.ssh/config, and the systemwide configuration file is probably either /etc/ssh/ssh_config or /usr/local/etc/ssh_config, depending on how you installed OpenSSH. (In either configuration file, you can set the parameters on a host-by-host basis, so that different options can apply to different target systems.)

Forwarding traffic with SSH

Not only can SSH forward your authentication requests, but it can forward your X Window System traffic as well. And SSH not only forwards the traffic, but also automatically sets X's DISPLAY variable on the remote system, thus simplifying your side of things substantially. Nobody likes messing with the DISPLAY variable, Magic Cookies, or any other part of remote X configuration. SSH can handle all of those at once. You just need to connect SSH using the -X option, or set ForwardX11 yes in the appropriate config files, and run your X application from the remote system's command prompt.

When using X, I always make sure to turn compression on with the -C option (Compression yes in the config file), as X is bandwidth-intensive, to put it politely. With compression on and a fast cipher such as blowfish (the default for recent versions of OpenSSH), Netscape is just about usable over a 10-Mb network. Without compression, it is rather painful, and I would certainly not recommend running any complex X application on less than 10 Mb.

Perhaps even more interesting is SSH's ability to forward arbitrary ports. Maybe you read mail on a server that you can also SSH into, and POP3 mail is running while secure POP3 is not. You therefore wish to forward a port on your workstation to the POP3 port on the server. Let's pick an arbitrary port, such as 2048. Configure your mail client (such as Netscape) to connect POP3 to port 2048 on localhost (that is, your workstation). Then you can run the following command: ssh -N -f -L 2048:mailserver.example.com:110 you@mailserver.example.com and you'll be able to connect securely.

What's happening here? -N tells SSH that you're not interested in running a command on the remote system, only in forwarding the connection, and -f tells SSH to go into the background once you've authenticated your connection. That way, after you've entered your passphrase (or once ssh-agent has authenticated you), you won't need to type more. The magical part is -L 2048:mailserver.example.com:110. That tells SSH to forward the local port 2048 to mailserver.example.com's port 110. (Port 110 is the standard POP3 port.) So you log in to the mail server, and then anything on local port 2048 is sent over the SSH connection to port 110 on the remote side, thus allowing you to safely retrieve your mail.

That method works very nicely for most protocols, including POP3, HTTP, and IMAP, just so long as you're able to tell the client server to use the local system and the appropriate port on your local system. For FTP, be sure to use passive FTP, because active FTP opens a second connection that will not be forwarded via the SSH port forwarding. Recent versions of OpenSSH, and Commercial SSH version 2, support SFTP, which works like FTP and uses SSH without the difficulty of port forwarding and passive mode.

Tricks and traps

Because SSH command syntax descends from the outdated and dangerous rsh syntax, several neat tricks are possible. My favorite is piping data through SSH to a program on the remote side. For example, I haven't configured printing on my laptop, as I can't be sure what device I'll want to print to. Instead, I can run cat /my/file/to/print.txt | ssh jon@my.remote.printhost lpr -Pwhich_printer, which pipes the file to the program lpr on the remote side of things. Essentially, anything after the destination host on the SSH command is passed as a command to the remote server. If disk space is tight, you can also use that trick to tar files to a storage place on a different system: tar cvf - source_directory | ssh user@remote_host 'cat > my-tar-file.tar'. The quotes are necessary to ensure that the remote system, not the local system, redirects output.

One trap associated with executing commands via SSH is that interactive programs tend to die reluctantly. That is generally because they expect a terminal to be available, and SSH does not allocate a terminal for commands that it does not believe to be interactive. For example, I keep nethack on one system and I execute it via a script that calls ssh jon@remote-system nethack. For a long time I did not understand why nethack failed to run properly, but then I discovered SSH's -t option, which forces allocation of a pseudo-TTY. Using ssh -t jon@remote-system nethack does the trick just fine.

Another trap is that dangerous configurations of the SSH client drop back to the notoriously insecure rsh program if the remote system does not have SSH installed. Your SSH client should print out an error message under those circumstances, and you would be ill-advised to ignore it. You can disable that behavior with a line reading FallBackToRsh No in your ssh configuration file.

Although this article hardly covers every facet of SSH, I hope you have learned a few new ways to use it. I've recently freed myself from the annoyance of typing passwords frequently. As a system administrator, I find that using SSH to run commands on remote systems helps me manage them efficiently, both in executing the same command on multiple systems (such as package upgrades) and in collecting remote data (such as running uptime and sending the output to a file). As a user, I find that, once ssh-agent is configured properly, my workday proceeds more smoothly.

ssh Manpage

NAME

SYNOPSIS

DESCRIPTION

OPTIONS

ENVIRONMENT VARIABLES

EXIT STATUS

FILES

scp(1) Manpage

NAME

SYNOPSIS

DESCRIPTION

OPTIONS

OPERANDS

EXIT STATUS

ATTRIBUTES

SEE ALSO

AUTHORS

How to Configure Clients

The clients ssh and scp are quite configurable, with many settings that can be changed to suit your whim. If you want to modify the behavior of these clients, three general techniques are at your disposal:

We now present a general overview of these three methods.

Environment Variables

Several client features are controlled by environment variables. For example, scp prints statistics about each file it transfers if the environment variable SSH_ALL_SCP_STATS is set. Environment variables may be set in your current shell by the standard methods:

# C shell family (csh, tcsh)
$ setenv SSH_ALL_SCP_STATS 1

# Bourne shell family (sh, ksh, bash)
$ SSH_ALL_SCP_STATS=1
$ export SSH_ALL_SCP_STATS

Alternatively, environment variables and values may be specified in a file. System administrators can set environment variables for all users in /etc/environment, and users can set them in ~/.ssh/environment (SSH1, OpenSSH) and ~/.ssh2/environment (SSH2). These files contain lines of the format:

NAME=VALUE

where NAME is the name of an environment variable, and VALUE is its value. The value is taken literally, read from the equals sign to the end of the line. Don't enclose the value in quotes, even if it contains whitespace, unless you want the quotes to be part of the value.

Command-Line Options

Command-line options let you change a client's behavior just once at invocation. For example, if you're using ssh1 over a slow modem connection, you can tell SSH1 to compress the data with the -C command-line option:

$ ssh1 -C server.example.com

ssh, scp, and most of their support programs, when invoked with the -h option, print a help message listing all of their command-line options. For example:

# SSH1, SSH2
$ ssh -h
$ ssh-keygen2 -h

Client Configuration Files

If you don't want to retype command-line options continually, configuration files let you change a client's behavior now and in the future, until you change the configuration file again. For example, you can enable compression for all clients you invoke by inserting this line into a client configuration file:

Compression yes

In a client configuration file, client settings are changed by specifying keywords and values.

In the example, the keyword is Compression and the value is yes. In SSH1 and OpenSSH, you may optionally separate the keyword and value with an equals sign:

Compression = yes

SSH2 doesn't support this syntax, however, so it can be easier always to use the "keyword <space> value" format to avoid confusion.

You may configure clients to behave differently for each remote host you visit. This can be done on the fly with command-line options, but for anything reasonably complex, you'll end up typing long, inconvenient command lines like:

$ ssh1 -a -p 220 -c blowfish -l sally -i myself server.example.com

Alternatively, you can set these options within a configuration file. The following entry duplicates the function of the command-line options above, collecting them under the name "myserver":

# SSH2, OpenSSH
Host myserver
 ForwardAgent no
 Port 220
 Cipher blowfish
 User sally
 IdentityFile myself
 HostName server.example.com

To run a client with these options enabled, simply type:

$ ssh1 myserver

Configuration files take some time to set up, but in the long run they are significant timesavers.

We've given you a peek at the structure of a configuration file: a Host specification, followed by a bunch of keyword/value pairs. In the coming sections, we continue this philosophy, defining the structure and general rules before explaining the meanings of keywords. Once the generalities are covered, we'll dive into specific keywords. Sound good? Let's go.

Keywords versus command-line options

As we cover the many configuration keywords, note that all can be supplied on the command line if desired. The -o command-line option exists for this purpose. For any configuration line of the form:

Keyword Value

you may type:

SSH1 and OpenSSH allow use of the equals sign (=) between the keyword and value, which allows you to omit the quotes on the command line: ssh -o Keyword=Value.

# SSH1, SSH2, OpenSSH
$ ssh -o "Keyword Value" ...

For example, the configuration lines:

User sally
Port 220

can be specified as:
# SSH1, SSH2, OpenSSH
$ ssh -o "User sally" -o "Port 220" server.example.com

SSH1 additionally permits an equals sign between the keyword and the value:

$ ssh1 -o User=sally -o Port=220 server.example.com

This example shows that the -o option may be specified multiple times on the command line. The option also works for scp in SSH1 and OpenSSH:

# SSH1, OpenSSH
$ scp -o "User sally" -o "Port 220" myfile server.example.com:

Another relationship between command-line options and configuration keywords is found in the -F option (SSH2 only). This option instructs an SSH2 client to use a different configuration file instead of ~/.ssh2/ssh2_config. For example:

$ ssh2 -F ~/.ssh2/other_config

Unfortunately there's no equivalent option for SSH1 or OpenSSH clients.

Global and local files

Client configuration files come in two flavors. A single, global client configuration file, usually created by a system administrator, governs client behavior for an entire computer. The file is traditionally /etc/ssh_config (SSH1, OpenSSH) or /etc/ssh2/ssh2_config (SSH2). (Don't confuse these with the server configuration files in the same directories.) Each user may also create a local client configuration file within his or her account, usually ~/.ssh/config (SSH1, OpenSSH) or ~/.ssh2/ssh2_config (SSH2). This file controls the behavior of clients run in the user's login session.

Note: The system administrator may change the locations of client configuration files, using the compile-time flag --with-etcdir or the serverwide keyword UserConfigDirectory.

Values in a user's local file take precedence over those in the global file. For instance, if the global file turns on data compression, and your local file turns it off, the local file wins for clients run in your account. We cover precedence in more detail soon.

Configuration file sections

Client configuration files are divided into sections. Each section contains settings for one remote host or for a set of related remote hosts, such as all hosts in a given domain.

The beginning of a section is marked differently in different SSH implementations. For SSH1 and OpenSSH, the keyword Host begins a new section, followed by a string called a host specification. The string may be a hostname:

Host server.example.com

an IP address:

Host 123.61.4.10

a nickname for a host:

Host my-nickname
or a wildcard pattern representing a set of hosts, where ? matches any single character and * any sequence of characters (just like filename wildcards in your favorite Unix shell):
Host *.example.com
Host 128.220.19.*

Some further examples of wildcards:

Host *.edu
Any hostname in the edu domain
Host a*				Any hostname whose
name begins with "a"
Host *1*				Any hostname
(or IP address!) with 1 in it
Host *				Any hostname or IP address

For SSH2, a new section is marked by a host specification string followed by a colon. The string, like the argument of Host, may be a computer name:

server.example.com:

an IP address:

123.61.4.10:

a nickname:

my-nickname:

or a wildcard pattern:

*.example.com:

128.220.19.*:

Following the host-specification line are one or more settings, i.e., configuration keywords and values, as in the example we saw earlier. The following table contrasts SSH1 and SSH2 configuration files:

SSH1, OpenSSH

SSH2

Host myserver
 User sally
 IdentityFile myself
 ForwardAgent no
 Port 220
 Cipher blowfish

myserver:
 User sally
 IdentityFile myself
 ForwardAgent no
 Port 220
 Ciphers blowfish

The settings apply to the hosts named in the host specification. The section ends at the next host specification or the end of the file, whichever comes first.

Multiple matches

Because wildcards are permitted in host specifications, a single hostname might match two or more sections in the configuration file. For example, if one section begins:

Host *.edu

and another begins:

Host *.harvard.edu

and you connect to server.harvard.edu, which section applies? Believe it or not, they both do. Every matching section applies, and if a keyword is set more than once with different values, the earliest value takes precedence.

Suppose your client configuration file contains two sections to control data compression, password authentication, and password prompting:

Host *.edu
 Compression yes
 PasswordAuthentication yes

Host *.harvard.edu
 Compression no
 PasswordPromptLogin no

and you connect to server.harvard.edu:

$ ssh server.harvard.edu

Notice that the string server.harvard.edu matches both Host patterns, *.edu and *.harvard.edu. As we've said, the keywords in both sections apply to your connection. Therefore, the previous ssh command sets values for keywords Compression, PasswordAuthentication, and PasswordPromptLogin.

But notice, in the example, that the two sections set different values for Compression. What happens? The rule is that the first value prevails, in this case, yes. So in the previous example, the values used for server.harvard.edu are:

Compression yes          The first of the Compression lines
PasswordAuthentication yes          Unique to first section
PasswordPromptLogin no          Unique to second section

and as shown in Figure below. Compression no is ignored because it is the second Compression line encountered. Likewise, if 10 different Host lines match server.harvard.edu, all 10 of those sections apply, and if a particular keyword is set multiple times, only the first value is used.

SSH1 client configuration file with multiple matches (SSH2 not shown)

While this feature might seem confusing, it has useful properties. Suppose you want some settings applied to all remote hosts. Simply create a section beginning with:

Host *

and place the common settings within it. This section should be either the first or the last in the file. If first, its settings take precedence over any others. This can be used to guard against your own errors. For example, if you want to make sure you never, ever, accidentally configure SSH sessions to fall back to the insecure rsh protocol, at the beginning of your configuration file put:

# First section of file
Host *
 FallBackToRsh no

Alternatively, if you place Host * as the last section in the configuration file, its settings are used only if no other section overrides them. This is useful for changing SSH's default behavior, while still permitting overrides. For example, by default, data compression is disabled. You can make it enabled by default by ending your configuration file with:

# Last section of file
Host *
 Compression yes

Voilá, you have changed the default behavior of ssh and scp for your account! Any other section, earlier in the configuration file, can override this default simply by setting Compression to no.

Making nicknames for hosts

Suppose your client configuration file contains a section for the remote host myserver.example.com :

Host myserver.example.com
 ...

One day, while logged onto ourclient.example.com, you decide to establish an SSH connection to myserver.example.com. Since both computers are in the same domain, example.com, you can omit the domain name on the command line and simply type:

$ ssh myserver

This does establish the SSH connection, but you run into an unexpected nuance of configuration files. ssh compares the command-line string "myserver" to the Host string "myserver.example.com", determines that they don't match, and doesn't apply the section of the configuration file. Yes, the software requires an exact textual match between the hostnames on the command line and in the configuration file.

You can get around this limitation by declaring myserver to be a nickname for myserver.example.com. In SSH1 and OpenSSH, this is done with the Host and HostName keywords. Simply use Host with the nickname and HostName with the fully qualified hostname:

# SSH1, OpenSSH
Host myserver
 HostName myserver.example.com
 ...

ssh will now recognize that this section applies to your command ssh myserver. You may define any nickname you like for a given computer, even if it isn't related to the original hostname:

# SSH1, OpenSSH
Host simple
 HostName myserver.example.com
 ...

Then you can use the nickname on the command line:

$ ssh1 simple

For SSH2, the syntax is different but the effect is the same. Use the nickname in the host specification, and provide the full name to the Host keyword:

# SSH2 only
simple:
 Host myserver.example.com
 ...

Then type:

$ ssh2 simple

Nicknames are convenient for testing new client settings. Suppose you have an SSH1 configuration for server.example.com:
Host server.example.com
 ...

and you want to experiment with different settings. You can just modify the settings in place, but if they don't work, you have to waste time changing them back. The following steps demonstrate a more convenient way:

  1. Within the configuration file, make a copy of the section you want to change:

    # Original
    Host server.example.com
     ...
    # Copy for testing
    Host server.example.com
     ...
  2. In the copy, change "Host" to "HostName":

    # Original
    Host server.example.com
     ...
    # Copy for testing
    HostName server.example.com
     ...

  3. Add a new Host line at the beginning of the copy, using a phony name, for example, "Host my-test":

    # Original
    Host server.example.com
    ...
    # Copy for testing
    Host my-test
     HostName server.example.com
     ...
  4. Setup is done. In the copy (my-test), make all the changes you want and connect using ssh my-test. You can conveniently compare the old and new behavior by running ssh server.example.com versus ssh my-test. If you decide against the changes, simply delete the my-test section. If you like the changes, copy them to the original section (or delete the original and keep the copy).

You can do the same with SSH2:

# Original
server.example.com:
 ...
# Copy for testing
my-test:
 Host server.example.com
 ...

Comments, indenting, and style

You probably noticed in the previous examples that we are using the # symbol to represent comments:

# This is a comment

In fact, any line beginning with # in the configuration file is treated as a comment and ignored. Likewise, blank lines (empty or containing only whitespace) are also ignored.

You might also have noticed that the lines following a host specification are indented:

# SSH1, OpenSSH
Host server.example.com
 Keyword1 value1
 Keyword2 value2

# SSH2 only
server.example.com:
 Keyword1 value1
 Keyword2 value2

Indenting is considered good style because it visually indicates the beginning of a new section. It isn't required, but we recommend it.

What Is Forwarding?

Forwarding is a type of interaction with another network application, as shown in Figure below.

SSH forwarding

SSH intercepts a service request from some other program on one side of an SSH connection, sends it across the encrypted connection, and delivers it to the intended recipient on the other side. This process is mostly transparent to both sides of the connection: each believes it is talking directly to its partner and has no knowledge that forwarding is taking place. Even more powerfully, SSH forwarding can achieve certain types of communication that are impossible without it.

Forwarding isn't a new concept. The basic operation of a terminal connection over a network (say, using telnet) is also a kind of forwarding. In a telnet connection, you sit on one end, your remote shell is on the other, and both sides operate as if directly connected by a serial cable. Nevertheless, sitting in the middle is a cooperating telnet client and server, forwarding bytes back and forth. SSH forwarding is much the same, except SSH plays fancy tricks with the data to add security.

We have also seen another type of SSH forwarding, agent forwarding. This let us create SSH connections from one computer, through a second computer, and onto a third using public-key authentication, but without installing our private key on the second machine. To accomplish this, an SSH server pretended to be an SSH agent, while transparently forwarding data to and from a remote agent. This paradigm holds true for TCP port forwarding and X forwarding, as the SSH server transparently masquerades as another network application

Port Forwarding

SSH uses TCP/IP as its transport mechanism, usually TCP port 22 on the server machine, as it encrypts and decrypts the traffic passing over the connection. We will now discuss a cool feature that encrypts and decrypts TCP/IP traffic belonging to other applications, on other TCP ports, using SSH. This process, called port forwarding, is largely transparent and quite powerful. Telnet, SMTP, NNTP, IMAP, and other insecure protocols running over TCP can be made secure by forwarding. IMAP server machine must be running an SSH server for port forwarding to provide real protection.

Our port forwarding example protects your IMAP connection but doesn't truly protect your email messages. Before reaching your IMAP server, the messages pass through other mail servers and may be intercepted in transit. For end-to-end email security, you and your correspondent should use tools such as PGP or S/MIME to sign and/or encrypt the messages themselves.

In short, with minimal configuration changes to your programs, SSH port forwarding protects arbitrary TCP/IP connections by redirecting them through an SSH session. Port forwarding can even pass a connection safely through a firewall if you configure things properly. Once you start securing your communications with port forwarding, you'll wonder how you ever got along without it. Here are examples of what you can do:

SSH port forwarding is a general proxying mechanism for TCP only. (See Sidebar "TCP Connections" for an overview of TCP concepts.) Forwarding can't work with protocols not built on TCP, such as the UDP-based DNS, DHCP, NFS, and NetBIOS, or with non-IP-based protocols, such as AppleTalk or Novell's SPX/IPX. We're being a little imprecise here. DHCP is entirely based on UDP, so SSH port forwarding can't do anything with it. The others, however, either use both TCP and UDP for different purposes or can sometimes be configured to run over TCP, though they generally use UDP. Nevertheless, in most common situations, SSH can't forward them.

Local Forwarding

In our earlier example, we had an IMAP server running on machine S, and an email reader on home machine H, and we wanted to secure the IMAP connection using SSH. Let's delve into that example in more detail.

IMAP uses TCP port 143; this means that an IMAP server will be listening for connections on port 143 on the server machine. To tunnel the IMAP connection through SSH, you need to pick a local port on home machine H (between 1024 and 65535) and forward it to the remote socket (S,143). Suppose you randomly pick local port 2001. The following command then creates the tunnel:

[3] You can also use ssh -L2001:S:143 S, substituting "S" for localhost, but we will discuss later why localhost is the better alternative when possible.

$ ssh -L2001:localhost:143 S

The -L option specifies local forwarding, in which the TCP client is on the local machine with the SSH client. The option is followed by three values separated by colons: a local port to listen on (2001), the remote machine name or IP address (S), and the remote, target port number (143).

The previous command logs you into S, as it will if you just type ssh S. However, this SSH session has also forwarded TCP port 2001 on H to port 143 on S; the forwarding remains in effect until you log out of the session. To make use of the tunnel, the final step is to tell your email reader to use the forwarded port. Normally your email program connects to port 143 on the server machine, that is, the socket (S,143). Instead, it's configured to connect to port 2001 on home machine H itself, i.e., socket (localhost,2001). So the path of the connection is now as follows:

  1. The email reader on home machine H sends data to local port 2001.

  2. The local SSH client on H reads port 2001, encrypts the data, and sends it through the SSH connection to the SSH server on S.

  3. The SSH server on S decrypts the data and sends it to the IMAP server listening on port 143 on S.

  4. Data is sent back from the IMAP server to home machine H by the same process in reverse.

Port forwarding can be specified only when you create an SSH connection. You can't add a forwarding to an existing SSH connection with any SSH implementation we know of, though there's nothing intrinsic to the SSH protocol that would prevent it, and it would sometimes be a useful feature. Instead of using the -L option to establish a local forwarding, you can use the LocalForward keyword in your client configuration file:

# SSH1, OpenSSH
LocalForward 2001 localhost:143
# SSH2 only
LocalForward "2001:localhost:143"

Note the small syntactic differences. In SSH1 and OpenSSH, there are two arguments: the local port number, and the remote socket expressed as host:port. In SSH2, the expression is just as on the command line, except that it must be enclosed in double quotes. If you forget the quotes, ssh2 doesn't complain, but it doesn't forward the port, either.

Our example with home machine H and IMAP server S can be set up like this:

# SSH1, OpenSSH
Host local-forwarding-example
 HostName S
 LocalForward 2001 localhost:143
# Run on home machine H
$ ssh local-forwarding-example


Local forwarding and GatewayPorts

In SSH1 and OpenSSH, by default, only the host running the SSH client can connect to locally forwarded ports. This is because ssh listens only on the machine's loopback interface for connections to the forwarded port; that is, it binds the socket (localhost,2001), a.k.a. (127.0.0.1,2001), and not (H,2001). So, in the preceding example, only machine H can use the forwarding; attempts by other machines to connect to (H,2001) get "connection refused." However, ssh for SSH1 and OpenSSH has a command-line option, -g, that disables this restriction, permitting any host to connect to locally forwarded ports:

# SSH1, OpenSSH
$ ssh1 -g -L<localport>:<remotehost>:<remoteport> hostname

The client configuration keyword GatewayPorts also controls this feature; the default value is no, and giving GatewayPorts=yes does the same thing as -g:

# SSH1, OpenSSH
GatewayPorts yes

There's a reason why GatewayPorts and -g are disabled by default: they represent a security risk.

Remote forwarding

A remotely forwarded port is just like a local one, but the directions are reversed. This time the TCP client is remote, its server is local, and a forwarded connection is initiated from the remote machine.

Continuing with our example, suppose instead that you are logged into server machine S to begin with, where the IMAP server is running. You can now create a secure tunnel for remote clients to reach the IMAP server on port 143. Once again, you select a random port number to forward (say, 2001 again) and create the tunnel:

$ ssh -R2001:localhost:143 H

The -R option specifies remote forwarding. It is followed by three values, separated by colons as before but interpreted slightly differently. The remote port to be forwarded (2001) is now first, followed by the machine name or IP address (localhost) and port number (143). SSH can now forward connections from (localhost,143) to (H,2001).

Once this command has run, a secure tunnel has been constructed from the port 2001 on the remote machine H, to port 143 on the server machine S. Now any program on H can use the secure tunnel by connecting to (localhost,2001). As before, the command also runs an SSH terminal session on remote machine H, just as ssh H does.

As with local forwarding, you may establish a remote forwarding using a keyword in your client configuration file. The RemoteForward keyword is analogous to LocalForward, with the same syntactic differences between SSH1 and SSH2:

# SSH1, OpenSSH
RemoteForward 2001 S:143

# SSH2 only
RemoteForward "2001:S:143"

For example, here's the preceding forwarding defined in an SSH2-format configuration file:

# SSH2 only
remote-forwarding-example:
 Host H
 RemoteForward "2001:S:143"
 
$ ssh2 remote-forwarding-example

You might think that the GatewayPorts feature discussed in the last section applies equally well to remote port forwardings. This would make sense as a feature, but as it happens, it isn't done. There would have to be a way for the client to communicate this parameter to the server for a given forwarding, and that feature hasn't been included in the SSH protocol. In SSH1 and SSH2, remotely forwarded ports always listen on all network interfaces and accept connections from anywhere. The OpenSSH server does accept the GatewayPorts configuration option, and it applies globally to all remote forwardings established by that server.

Trouble with Multiple Connections

If you use LocalForward or RemoteForward in your configuration file, you might run into a subtle problem. Suppose you have set up a section in your configuration file to forward local port 2001 to an IMAP server:

# SSH1 syntax used for illustration
Host server.example.com
 LocalForward 2001 server.example.com:143

This configuration works fine if you connect once:

$ ssh server.example.com

But if you try to open a second ssh connection to server.example.com at the same time-perhaps to run a different program in another window of your workstation-the attempt will fail:
$ ssh server.example.com
Local: bind: Address already in use

Why does this happen? Because your configuration file section tries to forward port 2001 again but finds that port is already in use ("bound" for listening) by the first instance of ssh. You need some way to make the connection but omit the port forwarding.

SSH1 (but not OpenSSH) provides a solution, the client configuration keyword ClearAllForwardings. From the name, you might think it terminates existing forwardings, but it doesn't. Rather, it nullifies any forwardings specified in the current ssh command. In the previous example, you can connect without forwardings to server.example.com with:

# SSH1 only
$ ssh1 -o ClearAllForwardings=yes server.example.com

The original tunnel, set up by the first invocation, continues to exist, but ClearAllForwardings prevents the second invocation from attempting to recreate the tunnel. To illustrate the point further, here's a rather silly command:

$ ssh1 -L2001:localhost:143 -o ClearAllForwardings=yes mymachine

The -L option specifies a forwarding, but ClearAllForwardings cancels it. This silly command is identical in function to:

$ ssh1 mymachine

ClearAllForwardings may also be placed in your client configuration file, of course. It seems more useful on the command line, however, where it can be used on the fly without editing a file.

Comparing Local and Remote PortForwarding

The differences between local and remote forwarding can be subtle. It can get a bit confusing to know which kind of forwarding to use in a given situation. The quick rule is look for the TCP client application.

If the TCP client application (whose connections you want to forward) is running locally on the SSH client machine, use local forwarding. Otherwise, the client application is on the remote SSH server machine, and you use remote forwarding.

The rest of this section is devoted to dissecting the forwarding process in detail and understanding where this rule comes from.

Common elements

Local and remote forwarding can be confusing because of overloaded terminology. In a given port forwarding situation, there are two clients and two servers lying around. We have the SSH client and server programs (e.g., ssh and sshd ), plus the TCP application's client and server programs whose connection you want to protect by port forwarding.

An SSH session has a direction of establishment. That is, you run an SSH client on one machine, and it initiates a session with an SSH server on another. Likewise, a forwarded connection has a direction of establishment: you run an application client on one machine, and it initiates a session with a service on another. These two directions may or may not match. This is the difference between local and remote forwarding. Let's introduce some terminology and provide some diagrams to make sense of this.

To begin with, we have an application client and server running on two hosts, A and B. The application server is listening on a well-known port W for incoming client connections. Without SSH, you can tell the application client that its server is on host B, port W. The client makes a direct connection to the server, and all application protocol data go in the clear over the network:

Direct client/server connection (no forwarding)

To protect the application protocol data by forwarding, you establish an SSH session between these two hosts. When setting up the SSH session, you select an unused port number P on the application client side (host A), and request SSH port forwarding from the socket (A,P) to the socket (B,W). Once the session is established, the SSH process on A is listening for incoming TCP connection requests on port P. Tell the application client that its server is on (A,P) instead of (B,W), and the stage is now set for port forwarding.

There are now two cooperating SSH processes with an established, encrypted SSH session between them; you don't yet distinguish between the SSH client and server. Inside that session, SSH creates multiple channels, or logical streams for carrying data. It uses channels to carry and distinguish the input, output, and error streams for an interactive login or remote command run via SSH, and similarly creates a new channel for each use of a port forwarding, to carry the forwarded data inside the protected SSH session.

As Figure above shows that now, when the application client tries to connect to its server, it connects instead to the listening SSH process (1). The SSH listener notices this and accepts the connection. It then notifies its partner SSH process that a new instance of this port forwarding is starting up, and they cooperate to establish a new channel for carrying the data for this forwarding instance (2). Finally, the partner SSH process initiates a TCP connection to the target of the port forwarding: the application server listening on (B,W) (3). Once this connection succeeds, the port forwarding instance is in place. The SSH processes cooperate to pass back and forth any data transmitted by the application client and server, over the channel inside the SSH session. This allows them to communicate and secures the application's activities on the network.

Local versus remote forwarding: the distinction

With this general framework in place, you can distinguish between local and remote forwarding. First we introduce some terms. In the generic port forwarding description in the last section, you saw that one SSH process listens for connections, while the other is ready to initiate connections in response to connections accepted on the other side, to complete the forwarded path. We call the first side the listening side of the SSH session with respect to this forwarding, and the other, the connecting side. For example, in the Figure above host A is the listening side, while host B is the connecting side. Note that these terms aren't mutually exclusive. Since a single SSH session may have multiple forwardings in place, the same side of a session may be the listening side for some forwardings, and simultaneously the connecting side for others. But with respect to any particular forwarding, it is one or the other.

Now, recall that in the last section we didn't label the SSH processes according to which was the SSH client and which the SSH server, but simply referred to two cooperating SSH processes. We do so now, and can state succinctly the local versus remote distinction:

So, as we said at the beginning of this section: use a local forwarding when the application client is on the local side of the SSH connection, and a remote forwarding when it's on the remote side.

Forwarding Off-Host

In all our discussions of port forwarding so far, the application client and server have been located on the machines on the ends of the SSH session. This is reflected in our always using "localhost" in naming the target socket of a forwarding:

$ ssh -L2001:localhost:143 server.example.com

Since the application server is located on the same machine as the connecting side of the SSH port forwarding, the target host can be "localhost." But the connections between the application client and the SSH listening side, and between the application server and the SSH connecting side, are themselves TCP connections. For convenience, TCP implementations allow programs to make connections between two sockets on the same host. The connection data is simply transferred from one process to another without actually being transmitted on any real network interface. However, in principle, either the application client or server-or both-could be on different machines, potentially involving as many as four hosts in a single forwarding:

Off-host port forwarding

Although this situation is possible, you generally don't want to do it for security reasons, namely privacy and access control.

Privacy

The complete path followed by forwarded data includes three TCP connections. But only the second connection, between the two SSH processes, is protected as a channel inside the SSH session. The other two connections are just simple TCP connections. Normally each of these is on a single host, and is therefore protected from network snooping or interference, so the entire forwarding path is secure. But if either of these two connections is between different hosts, its data will be vulnerable in transit.

Access control and the loopback address

The other security problem of off-host forwarding concerns the listening side. In short, the listening side of a forwarding has no access control, so intruders may gain access to it. To explain this problem, we must first discuss the loopback address of a host.

In addition to any physical network interfaces it may have, a host running IP has also has a virtual one called the loopback interface. This is a software construct, not corresponding to any network hardware. Nonetheless, the loopback appears and responds like a real interface. Under Unix, it is often named lo0 and is listed by ifconfig:

$ ifconfig -a
...
lo0: flags=849<UP,LOOPBACK,RUNNING,MULTICAST> mtu 8232
        inet 127.0.0.1 netmask ff000000

The loopback interface leads back to the host itself. A datagram "transmitted" on the loopback interface immediately appears as an incoming packet on the loopback interface and is picked up and processed by IP as being destined for the local host.

The loopback interface is always assigned the same IP address: 127.0.0.1, the loopback address,[4] and the local naming service provides the name "localhost" for that address. This mechanism gives a reliable way for processes to communicate with one another on the local host via IP, regardless of what IP addresses the host may have on real connected networks, or indeed if the host has no real network connections at all. You can always refer to your local host using the well-known loopback address.

[4] Actually, the entire network 127.0.0.0/8-comprising 24 million addresses-is reserved for addresses that refer to the local host. Only the address 127.0.0.1 is commonly used, although we have seen devices use a handful of others for special purposes, such as "reject" interfaces on a terminal server or router.

By design, a loopback address is local to its host. One machine can't contact the loopback address of another. Since the loopback address 127.0.0.1 is standard on all IP hosts, any connection to 127.0.0.1 leads a machine to talk to itself. (Plus, the loopback network isn't routed on the Internet.)

Listening on ("binding") an interface

When a host listens on a TCP port, it establishes a potential endpoint for a TCP connection. But the endpoints of a TCP connection are sockets, and a socket is an (address,port) pair, not a (host,port) pair. Listening must take place on a particular socket and thus be associated with a particular address, hence a particular interface on the host. This is called binding the interface. Unless otherwise specified, when asked to listen on a particular port, TCP binds all the host's interfaces and accepts connections on any of them. This is generally the right behavior for a server. It doesn't care how many network interfaces the local host has: it just accepts any connection made to its listening port, regardless of which host address was requested.

[5] Named after the Berkeley sockets library routine bind, commonly used to establish the association.

Consider, however, what this means in the case of SSH port forwarding. There is no authentication or access control at all applied to the listening side of a forwarding; it simply accepts any connection and forwards it. If the listening side binds all the host's interfaces for the forwarded port, this means that anyone at all with network connectivity to the listening host-possibly the whole Internet!-can use your forwarding. This is obviously not a good situation. To address it, SSH by default binds only the loopback address for the listening side of a forwarding. This means that only other programs on the same host may connect to the forwarded socket. This makes it reasonably safe to use port forwarding on a PC or other single-user machine but is still a security problem on multiuser hosts. On most Unix machines, for example, a knowledgeable user can connect to any listening sockets and see what's on them. Keep this in mind when using port forwarding on a Unix machine.

If you want to allow off-host connections to your forwarded ports, you can use the -g switch or GatewayPorts option to have the listening side bind all interfaces, as we did in an earlier example:

$ ssh1 -g -L P:S:W B

But be aware of the security implications! You may want to exercise more control over the use of forwarded ports in this situation by using TCP-wrappers, which we discuss later in this chapter.

Bypassing a Firewall

Let's tackle a more complicated example of port forwarding. As shown in the Figure below your home machine H talks to work machine W via a bastion host, B, and you want to access your work email from home. Machine W runs an IMAP server, and your home machine H has an IMAP-capable email reader, but you can't hook them up. Your home IMAP client expects to make a TCP connection directly to the IMAP server on W, but unfortunately that connection is blocked by the firewall. Since host B is inside the firewall, and it's running an SSH server, there should be some way to put all the pieces together and make the IMAP connection from H to W.

Port forwarding through a firewall

Port forwarding can solve this problem. As before, the IMAP server is on port 143, and we select a random local port number, 2001. This time, however, we use a slightly different command to set up forwarding:

# Executed on home machine H
$ ssh -L2001:W:143 B

This establishes an interactive SSH session from home machine H to bastion host B and also creates an SSH tunnel from local host H to the email server machine W. Specifically, in response to a connection on port 2001, the local SSH client directs the SSH server running on B to open a connection to port 143 on W, that is, socket W:143. The SSH server can do this because B is inside the firewall. If you configure your email reader to connect to local port 2001, as before, the communication path is now:

  1. The email reader on home machine H sends data to local port 2001.

  2. The local SSH client reads port 2001, encrypts the data, and sends it into the tunnel.

  3. The tunnel passes through the firewall, because it is an SSH connection (port 22) that the firewall accepts.

  4. The SSH server on bastion host B decrypts the data and sends it to port 143 on work machine W. This transmission isn't encrypted, but it's protected behind the firewall, so encryption isn't necessary. (Assuming you're not worried about snooping on your internal network.)

  5. Data is sent back from the IMAP server to home machine H by the same process in reverse.

You have now bypassed the firewall by tunneling the IMAP traffic through SSH.

Port Forwarding Without a Remote Login

It may happen that you'd like to forward a port via SSH but don't want an SSH login session to the remote host. For example, if you're using the IMAP forwarding example we've been harping on, you may want only to read email, not open an unnecessary terminal connection at the same time. With SSH2, this is simple: just provide the -f option to ssh2 in your port forwarding command:

# SSH2 only
$ ssh2 -f -L2001:localhost:143 server.example.com

or use the GoBackground keyword for the same effect:

# SSH2 only
GoBackground yes

As a result, ssh2 puts itself into the background and handles connections to the forwarded port 2001, and that is all. It doesn't create an interactive terminal session with standard input, output, and error channels. The -S option also avoids starting a terminal session but unlike -f, it doesn't put the session in the background (in other words, the -f option implies -S ):

# SSH2 only
$ ssh2 -S -L2001:localhost:143 server.example.com

The -f option is also supported by SSH1 and OpenSSH but its operation is different from that of SSH2. It is intended more for executing remote commands that don't require terminal interaction, such as graphical programs using X. Specifically:

For example, if X forwarding is turned on (which we'll discuss later), the following command puts itself into the background, popping up a graphical clock on your local display, with the clock program running on the remote host zwei.uhr.org :

# SSH1, OpenSSH
$ ssh -f zwei.uhr.org xclock

This is equivalent to the background command:

# SSH1, OpenSSH
$ ssh -n zwei.uhr.org xclock &

In contrast, SSH2 doesn't require a remote command when using the -f option. You may provide one as earlier, and ssh2 behaves in the same way as its SSH1 or OpenSSH counterparts:

$ ssh2 -f zwei.uhr.org xclock

but the remote command isn't necessary; you can set up a forwarding and put ssh2 into the background conveniently:

$ ssh2 -f -L2001:localhost:143 server.example.com

If you tried this with SSH1 or OpenSSH, you see:

# SSH1, OpenSSH
$ ssh -f -L2001:localhost:143 server.example.com
Cannot fork into background without a command to execute.

To get around the nuisance of providing an unwanted remote command, use one that does nothing for a long time, such as sleep :

# SSH1, OpenSSH
$ ssh -f -L2001:localhost:143 server.example.com sleep 1000000

One shot forwarding

When invoked with -f or GoBackground, ssh persists until you explicitly kill it with the Unix kill command. (You can find its pid with the ps command.) Alternatively, you can request one shot forwarding, which causes the client to exit when forwarding is over with. Specifically, the client waits indefinitely for the first forwarded connection. After that, when the number of forwarded connections drops to zero, the client exits.

One shot forwarding is accomplished easily in SSH2 with the -fo command-line option, a variation on -f (the "o" stands for "one shot "):

# SSH2 only
$ ssh2 -fo -L2001:localhost:143 server

One shot forwarding isn't directly supported by SSH1 or OpenSSH, but you can get the same effect with the following method:

  1. Set up the forwarding with ssh -f, and for the required remote command, use sleep with a short duration:

    $ ssh -f -L2001:localhost:143 server sleep 10
  2. Before the sleep interval expires, use the forwarded connection:

    $ ssh -p2001 localhost

Once the sleep command finishes, the first ssh tries to exit, but it notices a forwarded connection is in use and refuses to exit, printing a warning you can ignore:

Waiting for forwarded connections to terminate...
The following connections are open:
  port 2001, connection from localhost port 143

ssh waits until that connection ends and then terminates, providing the behavior of one shot forwarding.

The Listening Port Number

Earlier, we suggested selecting any unused port for the listening side of a forwarding. Port numbers are encoded in a 16-bit field and can have any value from 1 to 65535 (port is reserved). On multiuser operating systems such as Unix, ports 1 through 1023 are called privileged and are reserved for processes run by the superuser (user ID zero). If a nonprivileged process tries to bind a privileged port for listening, it will fail with an error message such as "insufficient permission."[6]

[6] Microsoft Windows and MacOS have no privileged port restriction, so any user can listen on any free port.

When setting up the listening side of a tunnel, you generally must select a port number between 1024 and 65535, inclusive. This is because an SSH program running under your user ID, not the superuser's, is responsible for listening on that port. If SSH reports that your chosen port is in already in use, just choose another; it shouldn't be hard to find a free one.

For the target side of the tunnel, you can specify any port number, privileged or not. You are attempting to connect to the port, not listen on it. In fact, most of the time the target side is a privileged port, since the most common TCP services have ports in the privileged range.

If you are the superuser on a machine with SSH clients, you can perform local forwarding with a privileged port. Likewise, you can forward a remote privileged port if your remote account has superuser privileges.

Some TCP applications hardcode the server port numbers and don't permit them to be changed. These applications aren't usable with port forwarding if the operating system has a privileged port restriction. For example, suppose you have an FTP client that's hardwired to connect to the server on the standard FTP control port, 21. To set up port forwarding, you have to forward the local port 21 to the remote port 21. But since port 21 is privileged, you can't use it as a listening port number unless you are the superuser. Fortunately, most Unix TCP-based programs let you set the destination port number for connections, and on PCs and Macs, there's no privileged port restriction.

Choosing the Target Forwarding Address

Suppose you want to forward a connection from your local machine to remote.host.net. The following two commands both work:

$ ssh -L2001:localhost:143 remote.host.net
$ ssh -L2001:remote.host.net:143 remote.host.net

The forwarded connection is made from the remote machine to either the loopback address or remote.host.net, and in either case, the connection stays on the remote machine and doesn't go over the network. However, the two connections are perceptibly different to the server receiving the forwarded connection. This is because the source sockets of the connections are different. The connection to localhost appears to come from source address 127.0.0.1, whereas the connection to remote.host.net is from the address associated with that name.

Most of the time this difference doesn't matter, but sometimes you must take it into account. The application server (e.g., the IMAP daemon) might be doing access control based on source address and not be configured to accept the loopback address. Or it might be running on a multihomed host and have bound only a subset of the addresses the host has, possibly not including the loopback address. Each of these situations is usually an oversight, but you might not be able to do anything about it. If you're getting "connection refused" from the connecting side of the forwarding, but you've verified that the server appears to be running and responding to normal clients, this might be the problem. If the server machine is running Unix, the command netstat -a -n should list all the network connections and listeners on that machine. Look for listeners on the relevant port, and the addresses on which they are listening.

Sometimes, the problem can be more acute if the server uses the source IP address itself as part of whatever protocol it's speaking. This problem crops up when trying to forward FTP over SSH. [Section 11.2]

In general, we recommend using localhost as the forwarding target whenever possible. This way, you are less likely to set up an insecure off-host forwarding by accident.

Termination

What happens to forwardings when an SSH connection terminates? The ports simply cease being forwarded; that is, SSH is no longer listening on them, and connection attempts to those ports get "connection refused."

What happens if you try to terminate an SSH session while it still has active forwarded connections? SSH will notice and wait for them to disconnect before stopping the session. The details of this behavior differ among implementations.

In SSH2, if you log out of a session that has an active forwarded connection, the session stays open but sends itself into the background:

remote$ logout
warning: ssh2[7021]: number of forwarded channels still open, forkedto background 
to wait for completion.
local$

The ssh2 process now waits in the background until the forwarded connections terminate, and then it exits. In contrast, with SSH1 and OpenSSH, if you disconnect a session with active forwardings, you get a warning, but the session stays in the foreground:

remote$ logout
Waiting for forwarded connections to terminate...
The following connections are open:
  port 2002, connection from localhost port 1465

To send it into the background and return to your local shell prompt, use the escape sequence return-tilde-ampersand:

~& [backgrounded]
local$

and as with SSH2, the connection exits only after its forwarded connections terminate. Be careful not to use the SSH ^Z escape for this purpose. That sends ssh into the background but suspended, unable to accept TCP connections to its forwarded ports. If you do this accidentally, use your shell's job control commands (e.g., fg and bg ) to resume the process.

The TIME_WAIT problem

Sometimes a forwarded port mysteriously hangs around after the forwarding SSH session has gone away. You try a command you've used successfully several times in a row and suddenly get an error message:

$ ssh1 -L2001:localhost:21 server.example.com
Local: bind: Address already in use

(This happens commonly if you're experimenting with port forwarding, trying to get something to work.) You know that you have no active SSH command listening on port 2001, so what's going on? If you use the netstat command to look for other listeners on that port, you may see a connection hanging around in the TIME_WAIT state:

$ netstat -an | grep 2001
tcp    0   0   127.0.0.1:2001   127.0.0.1:1472    TIME_WAIT

The TIME_WAIT state is an artifact of the TCP protocol. In certain situations, the teardown of a TCP connection can leave one of its socket endpoints unusable for a short period of time, usually only a few minutes. As a result, you cannot reuse the port for TCP forwarding (or anything else) until the teardown completes. If you're impatient, choose another port for the time being (say, 2002 instead of 2001) and get on with your work, or wait a short time for the port to become usable again.

Configuring Port Forwarding in the Server

We've seen several keywords and command-line options for configuring SSH clients for port forwarding, such as -L and -R. In addition, the SSH server can be configured for port forwarding. We'll cover compile-time, serverwide, and per-account configuration.

Compile-time configuration

You can enable or disable port forwarding at compile time with configure. It is enabled by default. For SSH1, the configure flags -disable-server-port-forwardings and -disable-client-port-forwardings turn off port forwarding capability for sshd1 and SSH1 clients, respectively. For SSH2, the single flag -disable-tcp-port-forwarding disables port forwarding for both clients and servers.

Port forwarding can be globally enabled or disabled in sshd. This is done with the serverwide configuration keyword AllowTcpForwarding in /etc/sshd_config. The keyword may have the value yes (the default, enabling forwarding) or no (disabling forwarding):

# SSH1, SSH2, OpenSSH
AllowTcpForwarding no

In addition, SSH2 has the following options:

# SSH2 only
AllowTcpForwardingForUsers
AllowTcpForwardingForGroups

The syntax of these is the same as for the AllowUsers and AllowGroups options. They specify a list of users or groups that are allowed to use port forwarding; the server refuses to honor port forwarding requests for anyone else. Note that these refer to the target account of the SSH session, not the client username (which is often not known).

F-Secure SSH1 Server supports the additional keywords AllowForwardingPort, DenyForwardingPort, AllowForwardingTo, and DenyForwardingTo for finer-grained control over forwarding. The two ...Port keywords let you control remote forwardings for given TCP ports, with support for wildcards and numeric ranges. For example, to permit remote forwardings for ports 3000, 4000 through 4500 inclusive, 5000 and higher, and any port number ending in 7:

# F-Secure SSH1 only
AllowForwardingPort 3000 4000..4050 >5000 *7

The ...To keywords are similar but control forwardings to particular hosts and ports (i.e., to particular sockets). Host and port specifications are separated by colons and use the same metacharacters as the ...Port keywords:

# F-Secure SSH1 only
DenyForwardingTo server.example.com:80 other.net:* yoyodyne.com:<1024

The permissible metacharacters/wildcards are shown in the following table:

Metacharacter

Meaning

Example

*

Any digit

300*

<

All values less than

<200

>

All values greater than

>200

..

Range of values (inclusive)

10..20

It's important to realize that the directives in this section don't actually prevent port forwarding, unless you also disable interactive logins and restrict what programs may be run on the remote side. Otherwise, knowledgeable users can simply run their own port-forwarding application over the SSH session. These settings alone might be a sufficient deterrent in a nontechnical community, but they won't stop someone who knows what she's doing.

Per-account configuration

In your account, you can disable port forwarding for any client that connects via a particular key. Locate the public key in your authorized_keys file and precede it with the option no-port-forwarding:

# SSH1, OpenSSH
no-port-forwarding ...key...

(SSH2 doesn't currently have this feature.) Any SSH client that authenticates using this key can't perform port forwarding with your SSH server.

The same remarks we just made about serverwide port forwarding configuration apply here: the restriction isn't really meaningful unless you further restrict what this key is allowed to do

X Forwarding

Now that you've seen general TCP port forwarding, we move to a new topic: forwarding of X protocol connections. X is a popular window system for Unix workstations, and one of its best features is its transparency. Using X, you can run remote X applications that open their windows on your local display (and vice versa, running local applications on remote displays). Unfortunately, the inter-machine communication is insecure and wide open to snoopers. But there's good news: SSH X forwarding makes the communication secure by tunneling the X protocol.

X forwarding also addresses some firewall-related difficulties. Suppose you're a system administrator with a set of exposed production machines on the other side of a firewall from you. You log into one of these machines using SSH, and want to run an graphical performance-monitoring tool, such as Solaris's perfmon, that uses the X Window System. You can't, though, because to do that, the external machine needs to make a TCP connection back to the internal machine you started on, and the firewall blocks it (as it should, since X is quite insecure). X forwarding solves this problem, permitting X protocol connections to pass through the firewall, securely tunneled via SSH.

Our discussion begins with a brief overview of X and then explains the details of X forwarding. In addition to explaining how to use X forwarding, we also expose the internals of X authentication and how it interacts with SSH, as well as other technical topics.

Sidebar 2. VNC Forwarding: An Alternative to X Forwarding

X forwarding is problematic from a security point of view, for the same reason as X itself. As we will see, the design of X means that remote programs must make separate network connections back to the user; this requires yet another layer of authentication and authorization, complicating the situation and opening an avenue of attack. SSH X forwarding tries to secure this as much as possible, but it may still be unacceptable in some environments.

An alternative technique is to use Virtual Network Computing (VNC) over SSH. VNC is free software developed by AT&T Laboratories in the United Kingdom, which provides remote GUI access for Unix and Windows platforms. With VNC, you can open a window on your Unix machine running X and have the desktop of a remote Windows machine appear there, so you can operate the Windows box remotely. Conversely, you can run the VNC client on a Windows machine and connect to a remote X display running on a Unix host. Since VNC involves only a single outbound connection, it is easier and safer to tunnel through SSH than X. You can find out more about VNC (and download the software) at:

http://www.uk.research.att.com/vnc/

The X Window System

The X Window System, or X, is the most widely used graphical display system for Unix machines. Like SSH, X has clients and servers. X clients are windowing application programs, such as terminal emulators, paint programs, graphical clocks, and so forth. An X server is the underlying display engine that processes requests from X clients, communicating via a network protocol called the X protocol. A machine typically runs a single X server but possibly many X clients.

Most important to our discussion, X supports sophisticated window management over a network. X clients can open windows not only on their local machine but also on other computers on the network, whether they are down the hall or across the globe. To accomplish this, an X client makes a network connection to a remote X server and carries on a conversation, using the X protocol to draw on the remote screen, receive remote keyboard events, learn the remote mouse location, and so on. This obviously requires some type of security, which we discuss soon.

A central concept of X is the display, an abstraction for the screen managed by an X server. When an X client is invoked, it needs to know which display to use. Displays are named by strings of the form HOST:n.v, where:

For example, on the machine server.example.com, display 0, visual 1 is represented by the display string "server.example.com:0.1".

Under Unix, most X client programs let you specify the display string in two ways: the -d or -display command-line option, or the environment variable DISPLAY. For example, to run the X client program xterm on the only X display of the workstation anacreon, use the command-line option:

$ xterm -d anacreon:0 &

or the environment variable:

$ setenv DISPLAY anacreon:0
$ xterm &

X is a large, deep software product whose documentation fills a dozen O'Reilly books. We've barely scratched the surface with our explanation, but you've now seen enough to understand X forwarding.

How X Forwarding Works

Although X clients can communicate with remote X servers, this communication isn't secure. All interactions between the X client and server, such as keystrokes and displayed text, can be easily monitored by network snooping because the connection isn't encrypted. In addition, most X environments use primitive authentication methods for connecting to a remote display. A knowledgeable attacker can get a connection to your display, monitor your keystrokes, and control other programs you're running.

Once again, SSH comes to the rescue. An X protocol connection can be routed through an SSH connection to provide security and stronger authentication. This feature is called X forwarding.

X forwarding works in the following way. An SSH client requests X forwarding when it connects to an SSH server (assuming X forwarding is enabled in the client). If the server allows X forwarding for this connection, your login proceeds normally, but the server takes some special steps behind the scenes. In addition to handling your terminal session, it sets itself up as a proxy X server running on the remote machine and sets the DISPLAY environment variable in your remote shell to point to the proxy X display:

syrinx$ ssh sys1
Last login: Sat Nov 13 01:10:37 1999 from blackberry
Sun Microsystems Inc.   SunOS 5.6       Generic August 1997
You have new mail.
sys1$ echo $DISPLAY 
sys1:10.0
sys1$ xeyes
The "xeyes" X client appears on the screen
X forwarding

The DISPLAY value appears to refer to X display #10 on sys1, but there's no such display. (In fact, there might be no true displays on sys1 at all.) Instead, the DISPLAY value points to the X proxy established by the SSH server, i.e., the SSH server is masquerading as an X server. If you now run an X client program, it connects to the proxy. The proxy behaves just like a "real" X server, and in turn instructs the SSH client to behave as a proxy X client, connecting to the X server on your local machine. The SSH client and server then cooperate to pass X protocol information back and forth over the SSH pipe between the two X sessions, and the X client program appears on your screen just as if it had connected directly to your display. That's the general idea of X forwarding.

X forwarding can even solve the firewall problem mentioned earlier, as long as the firewall permits SSH connections to pass through. If a firewall sits between your local and remote machines, and you run an X client on the remote machine, X forwarding tunnels the X connection through the firewall's SSH port to the local machine. Therefore, the X client's windows can open on your local display. If X forwarding isn't present, the firewall blocks the connection.

Some aspects of X forwarding probably sound familiar from our earlier explanation of port forwarding. In fact, X forwarding is just a special case of port forwarding for which SSH has special support.

Enabling X Forwarding

X forwarding is on by default in SSH1 and SSH2, but off in OpenSSH. If you need to enable or disable X forwarding for your clients, here's how to do it. Unlike general port forwarding, which requires you to fiddle with TCP port numbers, X forwarding has only an on/off switch. In your SSH client configuration file, use the keyword ForwardX11 with a value yes (the default, to enable) or no (to disable):

# SSH1, SSH2, OpenSSH
ForwardX11 yes

On the command line, you may also use -x to disable X forwarding:

# SSH1, SSH2, OpenSSH
$ ssh -x server.example.com

SSH2 and OpenSSH enables X forwarding with the following options:
# SSH2 only
$ ssh2 +x server.example.com

# OpenSSH only
$ ssh -X server.example.com

Configuring X Forwarding

The behavior of X forwarding can be modified through compile-time configuration, serverwide configuration, and per-account configuration.

Compile-time configuration

SSH1 and SSH2 can be compiled with or without X support. The compile-time flags --with-x and -without-x make this determination:

# SSH1, SSH2
$ configure ... --without-x ...

In addition, if you compile with X support, you may set the default behavior for X forwarding. In SSH1, you can enable or disable forwarding by default in the client and the server separately, using the compile-time flags -enable-client-x11-forwarding (or -disable-client-x11-forwarding) and -enable-server-x11-forwarding (or -disable-server-x11-forwarding):

# SSH1 only
$ configure ... --disable-server-x11-forwarding ...

In SSH2, you can enable or disable all X forwarding by default with -enable-X11-forwarding or -disable-X11-forwarding:

# SSH2 only
$ configure ... --enable-X11-forwarding ...

Remember, enable/disable flags simply set the default behavior. You can override these defaults with serverwide and per-account configuration.

Serverwide configuration

The serverwide configuration keywords X11Forwarding (SSH1, SSH2, OpenSSH) and its synonyms ForwardX11 (SSH2) and AllowX11Forwarding (SSH2) enable or disable X forwarding in the SSH server. By default, it is enabled:

# SSH1, SSH2, OpenSSH
X11Forwarding no

# SSH2 only: either will work
ForwardX11 no
AllowX11Forwarding no

The X11DisplayOffset keyword lets you reserve some X11 display numbers so sshd can't use them. This keyword specifies the lowest display number SSH may use, preventing sshd from clashing with real X servers on the lower-numbered displays. For example, if you normally run actual X servers on displays and 1, set:

# SSH1, OpenSSH
X11DisplayOffset 2

The XAuthLocation keyword specifies the path to the xauth program, which manipulates authorization records for X. We describe this keyword later, after we discuss xauth.

# SSH1, OpenSSH
XAuthLocation /usr/local/bin/xauth

Per-account configuration

In your SSH1 or OpenSSH authorized_keys file, you may disallow X forwarding for incoming SSH connections that use a particular key for authentication. This is done with the option no-X11-forwarding:

# SSH1, OpenSSH
no-X11-forwarding ...rest of key...

X Authentication

We've mentioned in passing that X performs its own authentication when X clients connect to X servers. Now we're going to dive into technical detail on the inner workings of X authentication, why it's insecure, and how SSH X forwarding builds on it to create a secure solution.

In most cases, X forwarding simply works, and you don't have to think about it. The following material is to aid your understanding and satisfy any intense cravings for tech talk (both yours and ours).

How X authentication works

When an X client requests a connection to an X server, the server authenticates the client. That is, the X server determines the client's identity to decide whether to allow a connection to the server's display. The current release of the X Window system (X11R6) provides two categories of authentication: host-based and key-based:

Host-based X authentication

The simpler method. Using the program xhost, you indicate a list of hosts that may connect to your X display. Notice that connections are authenticated only by hostname, not by username. That is, any user on a listed host may connect to your display.


Key-based X authentication

Uses the xauth program to maintain a list of X authentication keys, or display keys, for X clients. Keys are kept in a file, usually ~/.Xauthority, along with other data associated with the various displays the client wants to access. When an X client connects to a server requiring authentication, the client supplies the appropriate credentials for that display from the xauth data. If authentication is successful, the X client can then connect to the display managed by the X server.

Display keys are obtained from the X server in various ways depending on the environment. For example, if you start the server directly on the console of a machine using xinit or startx, these programs invoke an X server and insert a copy of the server's key directly into your xauth data. Alternatively, if you connect to a remote machine that runs the X Display Manager (XDM), the key is sent to your remote account when establishing your XDM session.

xauth and the SSH rc files

SSH has startup files that can be set to execute on the server side when a client logs in. These are the systemwide /etc/sshrc and the per-account ~/.ssh/rc. These can be shell scripts or any kind of executable program.

An important thing to note is that sshd runs xauth only to add the proxy display key if it doesn't run an rc program. If it does run an rc program, it feeds the key type and data to the program on a single line to its standard input, and it is up to the rc program to store the display key. This feature provides a way to customize handling the display key, in case just running xauth isn't the right thing to do in your situation.

Problems with X authentication

If you've used X, the authentication was probably transparent and seemed to work fine. Behind the scenes, however, the mechanism is insecure. Here are the major problems:

xhost is insecure

Once you give permission for a remote host to connect to your display, any user on that host can connect. As with the r-commands, this authentication method depends on the network address of the connecting host, which can be easy for an attacker to usurp.


Key transfer may be manual and insecure

Some remote-login protocols, such as telnet, don't assist with X authentication. If your display keys aren't available on a remote machine, you have to transfer them yourself, either manually or by automating the transfer, perhaps in your login script. This isn't only a nuisance but also insecure, since you're sending the key in plaintext over the network.


The most common key-based method, MIT-MAGIC-COOKIE-1, is insecure

Although it uses a random string of bits, or cookie, as the xauth display key, this key is transmitted in plaintext at the beginning of every connection, where it can be intercepted and read.

The remote host might not support your chosen X authentication method

X11R6 supports other, more secure authentication methods. SUN-DES-1 employs Sun's secure RPC system, XDM-AUTHORIZATION-1 uses DES, and MIT-KERBEROS-5 involves Kerberos user-to-user authentication.[7] Unfortunately, these methods are often not available in particular instances of the X software. Sometimes they aren't compiled into X installations due to cryptographic export restrictions; other times, the X version is too old to support the more secure methods.

[7] See the X11R6 Xsecurity (1) manpage for details on these methods. Also, remember that this is authentication only, not encryption. The contents of your X connection remain unencrypted and open to snooping or modification on the network.


If the remote host is insecure, your display key can be compromised

In the best scenario, where the X server supports strong authentication and your key can be copied securely to the remote machine, you still have to store your sensitive display key there. If that machine is untrustworthy, your key can be at risk. (SSH doesn't have this problem, since only your public key is stored on the SSH server machine.)

SSH and authentication spoofing

Through X forwarding, SSH provides transparent, secure authentication and key transfer for X sessions. This is done by a technique called authentication spoofing. Authentication spoofing involves a fake display key, which we call the proxy key, that authenticates access to the SSH X proxy server on the remote side. When relaying X traffic containing a key, SSH cleverly substitutes the real display key.

The players begin in the following positions. You are logged into a local machine with a local display. The local machine runs an X server and SSH clients. On the other side of the network connection, an SSH server is running on a remote machine, where you invoke X clients. The goal is for the remote X clients to appear on your local display by way of SSH.

First, you run a local SSH client, asking it to set up X forwarding. The SSH client requests X forwarding from the remote SSH server, and it also reads your local display key from your .Xauthority file.

Next, the SSH client generates a proxy key. This is a string of random data of the same length as your local display key. The SSH client then sends the proxy key and its key type (e.g., MIT-MAGIC-COOKIE-1) to the remote machine, and the SSH server runs the xauth program on your behalf to associate the proxy key with your local display. The stage is now set for X forwarding.

When you start a remote X client, your local SSH client connects to your local X display. It then watches for the first X protocol message sent over the forwarded connection and treats it specially. Specifically, the SSH client parses the message, finds the X authentication key inside it, and compares it to the proxy key. If the keys don't match, the SSH client rejects and closes the connection. Otherwise, if the keys match, the SSH client substitutes the real display key in place of the proxy key and relays the modified message to your local X server. The X server, blissfully unaware that a key switch has taken place, reads the display key and proceeds normally with X authentication. The forwarded X connection is now established.

X forwarding with authentication spoofing solves all but one of the X authentication problems we raised earlier:

xhost

X forwarding doesn't use xhost. (By the way, make sure to disable all xhost permissions when using X forwarding, or you will undermine the X security provided by SSH.)


Key transfer

SSH transfers the X display key automatically and runs xauth on your behalf to install it on the remote side. The transfer is secure since the key travels over the encrypted SSH connection.

MIT-MAGIC-COOKIE-1 insecurity

The key transmitted at the beginning of every X session is now encrypted, along with the rest of the X traffic, inside the SSH session. This greatly increases the operational security of this common X authentication scheme.

Untrustworthy remote hosts

With authentication spoofing, only the proxy key, not the true display key, is sent to the remote host. The proxy key is good only for connecting to your display through SSH, not for connecting to your display directly. As soon as your SSH session ends, the proxy key becomes useless. Since SSH sessions come and go, but some people leave their X sessions up (with the same key) for days, X forwarding can be a great improvement.

Improving authentication spoofing

The remaining problem with X forwarding is the possibility of unsupported X authentication mechanisms. The local side can use a more sophisticated authentication method a remote host might not support.

In theory, SSH X forwarding can solve this problem by always installing a proxy key of type MIT-MAGIC-COOKIE-1, no matter what local authentication method is actually in use. After the SSH client has checked the X client's key against the proxy key for a match, its client could then generate and substitute whatever local authenticator is required using the true authentication type and key.

Unfortunately, SSH implementations don't go this far. The server compares keys literally as bit strings, and the SSH client substitutes keys verbatim, regardless of the key types. As a result, if you use a stronger X authentication method such as XDM-AUTHORIZATION-1, sshd blindly compares an encrypted authenticator with the proxy key, rightly determine that they don't match, and invalidly rejects the connection. The failure is silent and mysterious; we wish the software would detect the presence of an unsupported mode and issue a warning when setting up the connection.

If SSH knew the details of all X authentication modes, it could check the proxy authenticators on one side and generate correct ones for the X server on the other. However, this is a significant development effort, though perhaps one could link SSH against the X11 libraries to obtain the necessary algorithms. SSH would also have to deal with differing key data lengths, constructing a new X message to hold the proxy key instead of copying it to an existing message.

It would also be useful if X forwarding could be used without authentication spoofing. Then you could arrange your own security for the connection by, say, using xhost to allow any connection from your local machine (and hence the SSH X proxy), while still applying key-based authentication to X connections originating from elsewhere. You can accomplish this with general port forwarding, as discussed in the next section, but direct support is more convenient.

Nonstandard X clients

X clients generally do X xauth-style authentication by virtue of having been linked against Xlib, the common X programming library. Occasionally, though, you run across particular X client programs that don't use Xlib and simply ignore authentication issues. Since you can't turn off SSH X authentication spoofing, you can't use such programs across SSH X forwarding; you get this message:

X11 connection requests different authenticationprotocol: 'MIT-MAGIC-COOKIE-1' 
vs. ''

You can, however, use a general port forwarding instead. For example:

foo% ssh -R6010:localhost:6000 bar
bar% setenv DISPLAY bar:10

Note that this bypasses the discipline imposed by X forwarding, of requiring xauth authentication on forwarded X connections. If your real X server is using xhost for access control, this port forwarding allows anyone on host foo to connect to your X server. Use this sort of thing with caution if you need to.

Further Issues

As we've said, X forwarding usually works fine without any special effort on your part. In some special situations, however, you might need to take some extra steps.

X server configuration

In order for X forwarding to work, your X server must accept the proxy X connections from your SSH client. This is sometimes not set up to begin with, because normal use doesn't require it. For example, if you're using an X server on a PC to access a remote Unix machine via XDM, you might never run local X clients at all, and they may not be allowed by default. You can run xhost +localhost to allow all connections from the your PC, while still applying key-based authentication to connections from other sources. This allows SSH-forwarded (and authenticated) connections to be accepted.

Setting your DISPLAY environment variable

SSH sets the DISPLAY variable automatically only if X forwarding is in effect. If you don't use X forwarding but want to use X on a remote machine you logged into via SSH, remember that you have to set the DISPLAY variable yourself. You should only do this when the both machines are on the same, trusted network, as the X protocol by itself is quite insecure.

Be careful not to set DISPLAY unintentionally! It is common for people to set the DISPLAY variable in a login command file or by other means. If you're not careful, this can make your X connections insecure without your noticing. If you use SSH to tunnel through a firewall that blocks normal X connections, then of course you'll notice because your X clients won't work. But if normal X connections are possible but undesirable, and X forwarding isn't in effect, your X programs will work but will (silently) not be secured. This is a good reason to block X traffic at the firewall if it presents a security risk or to configure your X server to accept connections only from the local host (the source of the SSH-forwarded X connections). If that's not feasible, you may want to put something like this in your login script:

#!/bin/csh
if ($?DISPLAY) then
   set display_host   = `expr "$DISPLAY" : '\(.*\):'`
   set display_number = `expr "$DISPLAY" : '.*:\([^.]*\)'`
   set my_host = `hostname`
   set result  = `expr '(' "$display_host" = "$my_host" ')' '&' '(' \
                 "$display_number" '>' "0" ')'`
   if ($result == 0) then
      echo "WARNING: X display $DISPLAY does not appear to be protected by SSH!"
      echo "unsetting DISPLAY variable just to be safe"
      unsetenv DISPLAY
   endif
endif

Shared accounts

If you share a single account among multiple people, you may have some trouble with X forwarding. For example, it is common for a group of sysadmins to share use of the root account. For each person to retain their own environment when using the root account, they may set their USER, LOGNAME, and HOME environment variables explicitly to reflect their personal accounts rather than the root account. If you use SSH to log into the root account with X forwarding turned on, though, it adds the proxy xauth key to root's .Xauthority file before the shell reads your login script and resets these environment variables. The result is that once you're logged in and try to use X, it fails: the X client looks in your .Xauthority file (because of the setting of your HOME variable), but the key isn't there.

You can deal with this problem by setting the XAUTHORITY variable to point to root's .Xauthority file, or by using code like the following in your login script to copy the needed key into your personal one:

if (($uid == 0) && ($?SSH_CLIENT) && ($?DISPLAY)) then
# If I do ssh -l root with X forwarding, the X proxy server's xauth key
# gets added to root's xauth db, not mine.  See if there's an entry for my
# display in root's xauth db...
  set key = `bash -c "xauth -i -f /.Xauthority list $DISPLAY 2> /dev/null"`
# ... and if so, copy it into mine.
  if ($? == 0) then
    xauth -bi add $key
    chown res ~res/.Xauthority >& /dev/null
  endif
endif

Location of the xauth program

Remember that sshd runs the xauth program on your behalf, to add the proxy key to your .Xauthority file on the remote side. The location of the xauth program is discovered when you configure the SSH package and compiled into the sshd executable. If xauth is subsequently moved, X forwarding won't work (ssh -v reveals this explicitly). For SSH1 and OpenSSH, the system administrator on the server side can use the serverwide configuration keyword XAuthLocation to set the path to the xauth program without having to recompile sshd1:

# SSH1, Open SSH
XAuthLocation /usr/local/bin/xauth

XAuthLocation can also appear in the client configuration file (OpenSSH only); the client uses xauth to get the local X display key.

X forwarding and the GatewayPorts feature

The GatewayPorts (-g) feature discussed earlier applies only to general port forwarding, not to X forwarding. The X proxies in SSH1, SSH2, and OpenSSH always listen on all network interfaces and accept connections from anywhere, though those connections are then subject to X authentication as described earlier. To restrict X client source addresses, use TCP-wrappers, which we discuss in the next section.

Forwarding Security: TCP-wrappers and libwrap

At several points in this chapter, we have talked about security issues and limitations of forwarding. So far, we've seen very little control over who can connect to a forwarded port. The SSH1 and OpenSSH default is to allow connections only from the local host, which is reasonably secure for a single-user machine. But if you need to allow connections from elsewhere, you have a problem, since it's all or nothing: to allow connections from elsewhere (using -g or GatewayPorts=yes), you must allow them from anywhere. And with SSH2 it's worse: forwarded ports will always accept connections from anywhere. X forwarding is in a slightly better position, since the X protocol has its own authentication, but you might still prefer to restrict access, preventing intruders from exploiting an unknown security flaw or performing a denial-of-service attack. SSH on the Unix platform provides an optional feature for access control based on the client address, called TCP-wrappers.

The term TCP-wrappers refers to software written by Wietse Venema. If it isn't already installed in your Unix distribution, you can get it at:

ftp://ftp.porcupine.org/pub/security/index.html

TCP-wrappers are a global access control mechanism that integrates with other TCP-based servers, such as sshd or telnetd. Access control is based on the source address of incoming TCP connections. That is, a TCP-wrapper permits or denies connections based on their origin, as specified in the configuration files /etc/hosts.allow and /etc/hosts.deny.:

There are two ways to use TCP-wrappers. The most common method, wrapping, is applied to TCP servers that are normally invoked by inetd. You "wrap" the server by editing /etc/inetd.conf and modifying the server's configuration line. Instead of invoking the server directly, you invoke the TCP-wrapper daemon, tcpd, which in turn invokes the original server. Then, you edit the TCP-wrapper configuration files to specify your desired access control. tcpd makes authorization decisions based on the their contents.

The inetd technique applies access control without having to modify the TCP server program. This is nice. However, sshd is usually not invoked by inetd, so the second method, source code modification, must be applied. To participate in TCP-wrapper control, the SSH server must be compiled with the flag -with-libwrap to enable internal support for TCP-wrappers. sshd then invokes TCP-wrapper library functions to do explicit access-control checks according to the rules in /etc/hosts.allow and /etc/hosts.deny. So in a sense, the term "wrapper" is misleading since sshd is modified, not wrapped, to support TCP-wrappers:

TCP-wrapper (libwrap) operation

TCP-wrappers Configuration

The access control language for TCP-wrappers has quite a few options and may vary depending on whose package you use and what version it is. We won't cover the language completely in this book. Consult your local documentation for a complete understanding: the manpages on tcpd (8), hosts_access (5), and hosts_options (5). We will just indicate some simple, common configurations.

The TCP-wrapper configuration is kept in the files /etc/hosts.allow and /etc/hosts.deny. These files contain patterns of the form:

service_1 [service_2 service_3 ...] : client_1 [client_2 client_3 ...]

Each pattern matches some (server,client) pairs, and hence may match a particular client/server TCP connection. Specifically, a connection between client C and server S matches this rule if some service service_i matches S, and some client_j matches C. (We explain the format and matching rules for these subpatterns shortly.) The hosts.allow file is searched first, followed by hosts.deny. If a matching pattern is found in hosts.allow, the connection is allowed. If none is found there, but one matches in hosts.deny, the connection is dropped. Finally, if no patterns match in either file, the connection is allowed. Nonexistence of either file is treated as if the file existed and contained no matching patterns. Note that the default, then, is to allow everything.

There is also an extended syntax, documented on the hosts_options (5) manpage. It may or may not be available, depending on how your TCP-wrapper library was built. It has many more options, but in particular, it allows tagging an individual rule as denying or rejecting a matching connection; for example:

sshd1 : bad.host.com : DENY

Using this syntax, you can put all your rules into the hosts.allow file, rather than having to use both files. To reject anything not explicitly allowed, just put ALL: ALL:DENY at the end of the file.

In a pattern, each service is a name indicating a server to which this pattern applies. SSH recognizes the following service names:

sshd

The main SSH server. This can be sshd, sshd1, sshd2, or whatever name you invoke the daemon under (its argv[0] value).

sshdfwd-x11

The X forwarding port.

sshdfwd-N

Forwarded TCP port N (e.g., forwarded port 2001 is service sshdfwd-2001).

Each client is a pattern that matches a connecting client. It can be:

Setup shown below allows connections to any service from the local host's loopback address, and from all addresses 192.168.10.x. This host is running publicly available servers for SSH1, POP, and IMAP, so we allow connections to these from anywhere, but SSH-2 clients are restricted to sources in another particular range of networks.

Sample /etc/hosts.allow File
#
# /etc/hosts.allow
#
# network access control for programs invoked by tcpd (see inetd.conf) or
# using libwrap.  See the manpages hosts_access(5) and hosts_options(5).

# allow all connections from my network or localhost (loopback address)
#
ALL : 192.168.10.0/255.255.255.0 localhost

# allow connections to these services from anywhere
#
ipop3d imapd sshd1 : ALL

# allow SSH-2 connections from the class C networks 
# 192.168.20.0, 192.168.21.0, ..., 192.168.27.0
#
sshd2 : 192.168.20.0/255.255.248.0

# allow connections to forwarded port 1234 from host blynken
sshdfwd-1234 : blynken.sleepy.net

# restrict X forwarding access to localhost
sshdfwd-x11 : localhost

# deny everything else
#
ALL : ALL : DENY

We allow connections to the forwarded port 1234 from a particular host, blynken.sleepy.net. Note that this host doesn't have to be on any of the networks listed so far but can be anywhere at all. The rules so far say what is allowed, but don't by themselves forbid any connections. So for example, the forwarding established by the command ssh1 -L1234:localhost:21 remote is accessible only to the local host, since SSH1 defaults to binding only the loopback address in any case. But ssh1 -g -L1234:localhost:21 remote is accessible to blynken.sleepy.net as well, as is either command using ssh2 instead (since ssh2 doesn't affect the localhost restriction and ignores the -g option). The important difference is that with this use of TCP-wrappers, sshd rejects connections to the forwarded port, 1234, from any other address.

The sshdfwd-x11 line restricts X-forwarding connections to the local host. This means that if ssh connects to this host with X forwarding, only local X clients can use the forwarded X connection. X authentication does this already, but this configuration provides an extra bit of protection.

The final line denies any connection that doesn't match the earlier lines, making this a default-to-closed configuration. If you wanted instead to deny some particular connections but allow all others, you would use something like this:

ALL : evil.mordor.net : DENY
telnetd : completely.horked.edu : DENY
ALL : ALL : ALLOW

The final line is technically not required, but it's a good idea to make your intentions explicit. If you don't have the host_options syntax available, you instead have an empty hosts.allow file and the following lines in hosts.deny :

ALL : evil.mordor.net
telnetd : completely.horked.edu

Notes About TCP-wrappers

Here are a few things to remember when using TCP-wrappers:

Supplement

Port Forwarding

To understand port forwarding, it's important to know some details about TCP, the Transmission Control Protocol. TCP is a fundamental building block of the Internet. Built on top of IP, it is the transport mechanism for many application-level Internet protocols such as FTP, Telnet, HTTP, SMTP, POP, IMAP, and SSH itself.

TCP comes with strong guarantees. A TCP connection is a virtual, full-duplex circuit between two communicating parties, acting like a two-way pipe. Either side may write any number of bytes at any time to the pipe, and the bytes are guaranteed to arrive unaltered and in order at the other side. The mechanisms that implement these guarantees, though, are designed to counter transmission problems in the network, such as routing around failed links, or retransmitting data corrupted by noise or lost due to temporary network congestion. They aren't effective against deliberate attempts to steal a connection or alter data in transit. SSH provides this protection that TCP alone lacks.

If an application doesn't need these guarantees about data integrity and order, or doesn't want the overhead associated with them, another protocol called User Datagram Protocol (UDP) often suffices. It is packet-oriented, and has no guarantees of delivery or packet ordering. Some protocols that run over UDP are NFS, DNS, DHCP, NetBIOS, TFTP, Kerberos, SYSLOG, and NTP.

When a program establishes a TCP connection to a service, it needs two pieces of information: the IP address of the destination machine and a way to identify the desired service. TCP (and UDP) use a positive integer, called a port number, to identify a service. For example, SSH uses port 22, telnet uses port 23, and IMAP uses port 143. Port numbers allow multiple services at the same IP address.

The combination of an IP address and a port number is called a socket. For example, if you run telnet to connect to port 23 on the machine at IP address 128.220.91.4, the socket is denoted "(128.220.91.4,23)." Simply put, when you make a TCP connection, its destination is a socket. The source (client program) also has a socket on its end of the connection, and the connection as a whole is completely defined by the pair of source and destination sockets.

In order for a connection attempt to a socket to succeed, something must be "listening" on that socket. That is, a program running on the destination machine must ask TCP to accept connection requests on that port and to pass the connections on to the program. If you've ever attempted a TCP connection and received the response "connection refused," it means that the remote machine is up and running, but nothing is listening on the target socket.

How does a client program know the target port number of a listening server? Port numbers for many protocols are standardized, assigned by the Internet Assigned Numbers Authority or IANA. (IANA's complete list of port numbers is found at http://www.isi.edu/in-notes/iana/assignments/port-numbers.) For instance, the TCP port number assigned to the NNTP (Usenet news) protocol is 119. Therefore, news servers listen on port 119, and newsreaders (clients) connect to them via port 119. More specifically, if a newsreader is configured to talk to a news server at IP address 10.1.2.3, it requests a TCP connection to the socket (10.1.2.3,119).

Port numbers aren't always hardcoded into programs. Many operating systems let applications refer to protocols by name, instead of number, by defining a table of TCP names and port numbers. Programs can then look up port numbers by the protocol name. Under Unix, the table is often contained in the file /etc/services or the NIS services map, and queries are performed using the library routines getservbyname() , getservbyport() , and related procedures. Other environments allow servers to register their listening ports dynamically via a naming service, such as the AppleTalk Name Binding Protocol or DNS's WKS and SRV records.

So far, we've discussed the port number used by a TCP server when a TCP client program wants to connect. We call this the target port number. The client also uses a port number, called the source port number, so the server can transmit to the client. If you combine the client's IP address and its source port number, you get the client's socket.

Unlike target port numbers, source port numbers aren't standard. In most cases, in fact, neither the client nor the server cares which source port number is used by the client. Often a client will let TCP select an unused port number for the source. (The Berkeley r-commands, however, do care about source ports. [Section 3.4.2.3]) If you examine the existing TCP connections on a machine with a command such as netstat -a or lsof -i tcp , you will see connections to the well-known port numbers for common services (e.g., 23 for Telnet, 22 for SSH), with large, apparently random source port numbers on the other end. Those source ports were chosen from the range of unassigned ports by TCP on the machines initiating those connections.

Once established, a TCP connection is completely determined by the combination of its source and target sockets. Therefore, multiple TCP clients may connect to the same target socket. If the connections originate from different hosts, the IP address portions of their source sockets will differ, distinguishing the connections. If they come from two different programs running on the same host, TCP on that host ensures they have different source port numbers.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Jun 22, 2019] Using SSH X session forwarding by Seth Kenlon

Jun 22, 2019 | www.redhat.com

Normally, you would forward a remote computer's X11 graphical display to your local computer with the -X option, but the OpenSSH application places additional security limits on such connections as a precaution. As long as you're starting a shell on a trusted machine, you can use the -Y option to opt out of the excess security:

$ ssh -Y 93.184.216.34

Now you can launch an instance of any one of the remote computer's applications, but have it appear on your screen. For instance, try launching the Nautilus file manager:

remote$ nautilus &

The result is a Nautilus file manager window on your screen, displaying files on the remote computer. Your user can't see the window you're seeing, but at least you have graphical access to what they are using. Through this, you can debug, modify settings, or perform actions that are otherwise unavailable through a normal text-based SSH session.

Keep in mind, though, that a forwarded X11 session does not bring the whole remote session to you. You don't have access to the target computer's audio playback, for example, though you can make the remote system play audio through its speakers. You also can't access any custom application themes on the target computer, and so on (at least, not without some skillful redirection of environment variables).

However, if you only need to view files or use an application that you don't have access to locally, forwarding X can be invaluable.

[Jun 18, 2019] Using SSH Port Forwarding as a Security Tool in Linux by Helder

Images deleted...
Jun 05, 2019 | linuxhandbook.com
Learn to configure SSH port forwarding on your Linux system. Remote forwarding is also explained.

Regular Linux users know about SSH , as it is basically what allows them to connect to any server remotely to be able to manage it via command line. However, this is not the only thing SSH can provide you for, it can also act as a great security tool to encrypt your connections even when there is no encryption by default.

For example, let's say you have a remote Linux desktop that you wish to connect via SMTP or email but the firewall on that network currently blocks the SMTP port (25) which is very common. Through a SSH tunnel you would simply connect to that particular SMTP service using another port by simply using SSH without having to reconfigure SMTP configuration to a different port and on top of that, gaining the encryption capabilities of SSH.

Configure OpenSSH for port forwarding

In order for OpenSSH Server to allow forwarding, you have to make sure it is active in the configuration. To do this, you must edit your /etc/ssh/ssh_config file.

For Ubuntu 18.04 this file has changed a little bit so, you must un-comment one line in it:

By default this line comes commented, you need to un-comment to allow forwarding

Once un-commented, you need to restart the SSH service to apply the changes:

restart SSH Daemon to apply changes recently done in its configuration

Now that we have our target configured to allow SSH forwarding, we simply need to re-route things through a port we know is not blocked. Let's use a very uncommonly blocked port like 3300:

So now we have done this, all traffic that comes to port 25 will automatically sent over to port 3300. From another computer or client we simply will connect to this server to its port 3300 and we will then be able to interact with it as it was SMTP server without any firewall restrictions to its 25 port, basically we simply re-routed its port 25 traffic to another (non blocked) one to be able to access it.

READ 3 Ways to List Users in Linux The other way around: Remote Forwarding

We talked about forwarding a local port to another port, but let's say you want to do it exactly opposite: you want to route a remote port or something you currently can access from the server to a local port.

To explain it easily, let's use an example similar to the previous one: from this server you access a particular server through port 25 (SMTP) and you want to "share" that through a local port 3302 so anyone else can connect to your server to the 3302 port and see whatever that server sees on port 25:

Summing up and some tips on SSH port forwarding

As you can see, this SSH forwarding acts like a very small VPN, because it routes things to given ports. Whenever you execute these commands, they will open SSH shells, as it understands you need to interact to the server via SSH. If you don't need this, it will be enough to simply add the "-N" option in them, so they will simply not open any shell.

Liked the article? Please share it and help us grow :)

About Helder Systems Engineer, technology evangelist, Ubuntu user, Linux enthusiast, father and husband.

[Nov 03, 2018] setting up your own VPN server

Nov 03, 2018 | linuxize.com

The simpler alternative is to route your local network traffic with an encrypted SOCKS proxy tunnel. This way, all your applications using the proxy will connect to the SSH server and the server will forward all the traffic to its actual destination. Your ISP (internet service provider) and other third parties will not be able to inspect your traffic and block your access to websites.

This tutorial will walk you through the process of creating an encrypted SSH tunnel and configuring Firefox and Google Chrome web browsers to use SOCKS proxy.

Prerequisites Set up the SSH tunnel

We'll create an SSH tunnel that will securely forwards traffic from your local machine on port 9090 to the SSH server on port 22 . You can use any port number greater than 1024 .

Linux and macOS

If you run Linux, macOS or any other Unix-based operating system on your local machine, you can easily start an SSH tunnel with the following command:

ssh -N -D 9090 [USER]@[SERVER_IP]

The options used are as follows:

Once you run the command, you'll be prompted to enter your user password. After entering it, you will be logged in to your server and the SSH tunnel will be established.

You can setup a SSH key-based authentication and connect to your server without entering a password.

Windows

Windows users can create an SSH tunnel using the PuTTY SSH client. You can download PuTTY here .

  1. Launch Putty and enter your server IP Address in the Host name (or IP address) field.
  2. Under the Connection menu, expand SSH and select Tunnels . Enter the port 9090 in the Source Port field , and check the Dynamic radio button.
  3. Click on the Add button as shown in the image bellow.
  4. Go back to the Session page to save the settings so that you do not need to enter them each time. Enter the session name in the Saved Session field and click on the Save button.
  5. Select the saved session and login to the remote server by clicking on the Open button.

    A new window asking for your username and password will show up. Once you enter your username and password you will be logged in to your server and the SSH tunnel will be started.

Configuring Your Browser to Use Proxy

Now that you have open the SSH SOCKS tunnel the last step is to configure your preferred browser to use it.

Firefox

The steps bellow are the same for Windows, macOS, and Linux.

  1. In the upper right hand corner, click on the hamburger icon to open Firefox's menu:
  2. Click on the ⚙ Preferences link.
  3. Scroll down to the Network Settings section and click on the Settings... button.
  4. A new window will open.
    • Select the Manual proxy configuration radio button.
    • Enter 127.0.0.1 in the SOCKS Host field and 9090 in the Port field.
    • Check the Proxy DNS when using SOCKS v5 checkbox.
    • Click on the OK button to save the settings.

At this point your Firefox is configured and you can browse the Internet thought your SSH tunnel. To verify it you can open google.com , type "what is my ip" and you should see your server IP address.

To revert back to the default settings go to Network Settings , select the Use system proxy settings radio button and save the settings.

There are also several plugins that can help you to configure Firefox's proxy settings such as FoxyProxy .

Google Chrome

Google Chrome uses the default system proxy settings. Instead of changing your operating system proxy settings you can either use an addon such as SwitchyOmega or start Chrome web browser from the command line.

To launch Chrome using a new profile and your SSH tunnel use the following command:

Linux :

/usr/bin/google-chrome \
    --user-data-dir="$HOME/proxy-profile" \
    --proxy-server="socks5://localhost:9090"

macOS :

"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
    --user-data-dir="$HOME/proxy-profile" \
    --proxy-server="socks5://localhost:9090"

Windows :

"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" ^
    --user-data-dir="%USERPROFILE%\proxy-profile" ^
    --proxy-server="socks5://localhost:9090"

The profile will be created automatically if it does not exist. This way you can run multiple instances of Chrome at the same time.

To confirm the SSH tunnel is working properly, open google.com , and type "what is my ip". The IP shown should be the IP address of your server.

Conclusion

You have learned how to set up an SSH SOCKS 5 tunnel and configure your browser to access the Internet privately and anonymously.

If you hit a problem or have a feedback, leave a comment below.

Recommended Links

Google matched content

Softpanorama Recommended

Top articles

Sites



Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D


Copyright © 1996-2018 by Dr. Nikolai Bezroukov. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) in the author free time and without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: November 13, 2018