Using multiple SSH daemons on a single host

About

The information in this article is mostly outdated since sshd has had support for multiple ListenAddress directives for a good while. However, some of the specific configuration differences mentioned in the article can only be obtained by running separate daemons (the Match directive in sshd_config does not support all configuration directives otherwise available in a stand-alone sshd_config).

In the past I have run into the situation where I needed to hook up 2 CentOS GNU/Linux servers to allow the transfer of a large data volume between the both of them. Both servers are internet facing systems and the obvious choice would have been to haul the data across via SCP (secure copy) using the public network interface. Unfortunately, all traffic originating from the public interface was being billed and the entire data transfer would have resulted in a rather hefty bill if I would have used the public internet as transport medium. An additional requirement stipulated that this kind of data transfer would need to happen on a regular basis which would make the use of CD/DVD’s rather cumbersome. Since both boxes had a second NIC, the obvious choice was to connect them via a simple cross cable and to use a non-routable IP range for these NICs. Here is a simplified picture of the network setup:

• eth0: public interface, wired to switch/router, internet facing (IP: 213.123.213.1 & 213.123.213.2)
• eth1: private interface, wired via cross-cable to other server (IP: 10.10.0.1 & 10.10.0.2)

By default, the standard SSHD configuration on CentOS will cause the SSHD to listen on all available network interfaces. In my case that was on

eth0

and

eth1

So that is all that would have been required to make the secure copy work across the private network. However, the big disadvantage of this approach is that applied security settings for the running SSH daemon are valid for both networks. Typically, I never allow password based authentication on a public server but rather use SSH private/public keys instead. Additionally, direct root logins are also never allowed. Having to deal with this elevated security turned out to be rather a nuisance because opening terminal sessions and copying files always would have to happen using SSH keys and non-root logins.

To reconcile security and operational requirements I decided to go for the following solution: to run 2 separate SSH daemons, one for each network interface. In my case I called the second SSH daemon serving the private NIC sshd-internal.

The rest of the topic describes the required configuration to get this up and running. All steps have to be replayed on both systems using the applicable IP addresses where necessary. Replace the IP addresses in the listings with IP addresses that are valid for your setup.

SSH daemon configuration

We start with creating a new sshd_config file in /etc/ssh:

# cd /etc/ssh
# cp sshd_config sshd_config-internal

In the sshd_config-internal configuration file we can relax the security settings a bit and we change the ListenAddress directive so that the sshd-internal will only listen on the private network interface:

# diff sshd_config sshd_config-internal
57c57
< PasswordAuthentication no
---
> #PasswordAuthentication yes
107c107
< #PidFile /var/run/sshd.pid
---
> PidFile /var/run/sshd-internal.pid
112c112
< Banner /etc/issue
---
> #Banner /some/path
122c122
< PermitRootLogin no
---
> PermitRootLogin yes
127c127
< ListenAddress 213.123.213.1
---
> ListenAddress 10.10.0.1

Additionally, we also change the ListenAddress directive in the default sshd_config file. This time to make sure the standard SSH daemon will only listen on the public network interface:

# diff sshd_config sshd_config-internal (continued)
< #ListenAddress 0.0.0.0
---
> ListenAddress 213.123.213.1

In order to completely isolate both SSHD binaries from their start/stop sequence in /etc/rc.d/init.d we make a copy of the current sshd binary:

# which sshd
/usr/sbin/sshd

# ln -s /usr/sbin/sshd /usr/sbin/sshd-internal

# ll /usr/sbin/sshd*
384 -rwxr-xr-x 1 root root 383148 Mar 21 21:49 /usr/sbin/sshd

0 lrwxrwxrwx 1 root root      4 Jul 16 21:47 /usr/sbin/sshd-internal -> sshd

Next, we tackle the start/stop sequence as we want to able to independently start & stop both daemons. We therefore copy the default SSH daemon start file:

# cd /etc/rc.d/init.d/
# cp sshd sshd-internal

Then we need to edit the new sshd-internal start/stop script to reflect paths to the sshd-internal instance:

# diff sshd sshd-internal
12,13c12,13
< # config: /etc/ssh/sshd_config
< # pidfile: /var/run/sshd.pid
---
> # config: /etc/ssh/sshd_config-internal
> # pidfile: /var/run/sshd-internal.pid
19c19
< [ -f /etc/sysconfig/sshd ] && . /etc/sysconfig/sshd
---
> [ -f /etc/sysconfig/sshd-internal ] && . /etc/sysconfig/sshd-internal
22c22
< prog="sshd"
---
> prog="sshd-internal"
26c26
< SSHD=/usr/sbin/sshd
---
> SSHD=/usr/sbin/sshd-internal
30c30
< PID_FILE=/var/run/sshd.pid
---
> PID_FILE=/var/run/sshd-internal.pid

Notice that I did not only change the path to the new sshd binary and configuration file but also to a new PID file. The new *sshd-internal *init script will try to source a configuration file in /etc/sysconfig/sshd-internal. In this configuration file we will tell the SSH daemon to start up with a different SSHD configuration file from the default /etc/ssh/sshd_config:

# cat /etc/sysconfig/sshd-internal
OPTIONS="-f /etc/ssh/sshd_config-internal"

If your system has PAM enabled you will also need to provide a separate PAM configuration file for the new sshd-internal service. To keep things easy we are just going to link to the default sshd configuration file for PAM:

# cd /etc/pam.d
# ln -s sshd sshd-internal

Finally we can (re-)start both SSH daemons with their split setup:

# service sshd restart
# service sshd-internal start

The advantage of providing a separate sshd-internal binary through a symbolic link to the original sshd is that both running SSH daemons will always use the same application version. This is also true after an update of its RPM package for example.

Make sure your firewall disallows traffic towards the private network interface from the outside world.