SSH Tunnels - and other SSH Tricks

SSH Tunnel

The basic command for setting up a tunnel is as follows:

1 ssh -L <local_port>:<destination_host>:<destination_port> <remote_user>@<remote_host>

The following commandline snippets show how you can tunnel through ssh in order to ftp a file onto a remote box and how to access a web port on a remote server locally.

1 ssh -L 1234:ftp_server:21 middleman@188.88.88.88
2 ssh -L 80:middleman:8080 middleman@188.88.88.88

In the first example we set up the tunnel through a middleman server (jump host) that we have access to, and that does have access to port 21 on ftp_server. We can then ftp to and from ftp_server by connecting the FTP client to port 1234 on our local machine.

The second example shows how we can expose a port (8080) on a server we have ssh access to locally. This is useful if we have code deployed to a local machine that needs access to a service on a test server, but access restrictions stop us connecting to the relevant port directly. In this case we do have ssh access to the remote server we are trying to access the service on, we just don't have access to the port we need (8080 in this case). The tunnel forwards port 80 on our machine to port 8080 on the test server.

SSH Reverse Tunnel

Reverse tunnels solve the opposite problem. Your computer is running a version of an API that you are developing, and you want to temporarily point an application on a test server at the API on your computer. Due to the way the networks and firewalls are configured, there is no access from the test server to port 80 on your machine. This is where reverse tunnels come in.

1 ssh -R <remote_port>:destination_host:destination_port <remote_user>@<remote_host>

For the above example, this may be something like:

1 ssh -R 1234:localhost:80 sourceuser@jumphost

The above sets up a reverse tunnel from the server 'jumphost' (a box which you have ssh access to from your laptop and which the test server has access to port 1234) to your laptop using port 1234. This allows you to hook up the remote application on the test server to consume your development endpoint by configuring it to point at port 1234 on 'jumphost'.

Other examples may be to allow you to ssh from one box to another when they are on separate NAT'd networks via a host or hosts available to both over ssh. In such a case you may need a combination of reverse and forward tunnels to provide the connectivity you want. This sort of tunnel chaining can also be used to set up ad hoc VPN's for services - all data over the tunnel is (by nature of being SSH) encrypted.

Dealing with SSH Timeouts

Quite often policy demands that your SSH sessions are configured to time out after a period of inactivity. This is obviously not very useful if you need to set up tunnels. A common technique is to set a program like top or watch "date" or similar to keep the session reporting data over the connection and hence keeping the session alive.

If doing something like this, it is a good idea to do so on top of a screen session, since you can easily then open a new screen if you need to do work on the box you are ssh-ing onto.

Managing Keys

SSH Keys provide a solution to having to log in to a server with your credentials every time you want to ssh onto a box. The idea is that you generate a private/public key pair and copy the public one onto the servers you want to access. When you ssh onto those machines, the ssh protocol looks up your default key (or uses one you have specified on the command line) and sends it over the wire in place of a password.

Generating a key is simple:

1 ssh-keygen

This command will prompt you for a filename for the (private) key, and a passphrase to keep the key secure, and then generates a public/private key pair. The private key should always remain on your machine, but the public key may be copied wherever you need it to be. Github for example requires an ssh key in order for you to connect to it via the git protocol. Note that it is easiest to accept the default key file, as most ssh tools assume that that is where your key is. If you choose to put it elsewhere, you will have to specify the key to the command (typically with the -i option) or set it in your ssh config file (see below).

Enabling key based authentication is simply a matter of adding your public key to the ~/.ssh/authorized_keys file on the server you wish to authenticate on. This should be the text content of the public key on its own line in the authorized_keys file. A command which makes this very easy is the following:

1 ssh-copy-id user@host

This will put your key in the authorized_keys file on the host machine - but this time will ask you to provide your password. Subsequent ssh commands will be allowed straight in.

Managing SSH Servers

Unless you have a solution such as Kerberos based authentication for your servers, it can be hard to remember all of the hostnames of the servers you deal with on a daily basis and the usernames you need to log in with, so a way of managing them is desirable. This is where the ~/.ssh/config file comes in. It has a pretty simple format:

1 Host dev-box
2 HostName lnx07zz01.mydomain
3 User antroy
4 
5 Host prod
6 HostName lnx07zz01.mydomain
7 User antroy
8 IdentityFile ~/.ssh/prod_key

The examples above include a simple entry that associates an alias for the user 'antroy' on the given host. The second example also provides a different ssh key than the default (~/.ssh/id_rsa.) This allows you to specify different keys for different machines to help prevent the loss of a private key compromising all of the machines you log in to.

Managing SSH Keys

Having a private ssh key can be useful, but as noted above a potential security risk. If you lose your private key anyone has access to any server that has your public key in its authorized_keys file. This risk can be mitigated to a large extent by securing the private key with a passphrase when the key is generated. Now, to use your private key you will need to type the passphrase. But wait a minute...

Old world

I need to log in to many different servers with a password. If I'm lucky all of my credentials are the same, but possibly not.

Intermediate world

I set up a key pair with no passphrase, and now I can log into all of those boxes without a password. Sweet. But insecure...

New world

I set a passphrase on my private key. We are secure again. But I again have to type in a passphrase every time I ssh onto a server. Only very marginally better than the old world since the password is at least the same for every box - though I have to go to the trouble of installing my key on each box.

Enter the ssh-agent program.

SSH Agent and Storing Unlocked SSH Keys

Clearly someone had realised the futility of ssh key-based access to servers when the private keys are passphrase protected and so part of the suite of ssh tools is the ssh-agent program. This needs to be started when the users shell starts (typically the xsession or login session) and the keys added to the agent. The agent will prompt for passphrases for those keys once at the start of your session, and from then on will serve the unlocked keys to the ssh program.

On recent versions of Ubuntu the xsession starts with ssh-agent running. Other distros may differ and you may need to have your login startup file run ssh-agent. To add your keys and ensure that you get asked for the passphrase the first time you use the key, then add the following to your .bashrc or .profile:

1 export SSH_ASKPASS=/usr/bin/ssh-askpass
2 ssh-add </dev/null

Note - ssh-add will add your default keys to the agent. i.e. ~/.ssh/id_rsa and friends. To add other keys explicitly, they can be specified as a parameter to ssh-add. e.g.

1 ssh-add ~/.ssh/dev_id ~/.ssh/prod_id </dev/null

World as it should be

I set a passphrase on my private key and enter that passphrase when my computer starts up - and that passphrase is remembered by the ssh-agent for the duration of my session. Even better - if you have a password manager running (from Gnome or KDE for example) it will remember the passwords you give to the ssh-agent, so you only need to authenticate once for your password manager in order for the passphrases for all of your keys to be stored.

The ssh suite of programs are useful, versatile and should be a part of any developer or operations engineer's toolbox.