Remote access for embedded systems behind a firewall

When deploying embedded systems, wouldn’t it be nice if you could easily set up remote access for troubleshooting and updates? With Embedded Linux systems, this is quite easy to do. This article will explore how to use SSH to set up remote access to an embedded system behind a firewall without using VPN or special network setup and presents a few tips for making this setup work well.

The Problem

With today’s complex embedded systems, testing and deployment can be a challenge. During the initial testing and roll-out, a new system may need to be carefully monitored to assure it is operating correctly. Embedded Systems programming is different than application programming in that often you are interfacing with external I/O. The behavior of things external to the embedded system are often very difficult to model during development; therefore time must be spent in the field testing the system in real-world situations. Being able to remotely access the system during testing is very convenient.

As with most networked computer systems, networked embedded systems are typically deployed on a network behind a firewall. It is very easy to log in and administer any Linux system you have direct network access to — the challenge is getting to a system behind a firewall.

Options for remote Access

There are several general options for remote access to networks:

VPN
Port forwarding at the firewall
PC remote access technologies. These are typically single session type connections that are initiated by the PC user.

The disadvantage of the above technologies is they require involvement from the IT department or users of the remote network. BEC is currently involved in the deployment of a system that uses embedded computers in six different buildings. Each building has its own firewalled network. Getting remote access to all of these networks is a big task and everyone is busy. An ideal solution would be a way to communicate directly with each embedded computer from a central location, which brings us to SSH …

SSH

SSH (Secure Shell) is a set of standards and an associated network protocol that is used to establish a secure channel between a local and remote computer. SSH encrypts all traffic (including passwords) and is generally considered to be very secure if used properly. Some uses for SSH are:

remote terminal access.
transfer files between computers.
tunnelling and port forwarding between computers

The fact that SSH can be used to forward ports over an encrypted tunnel is what makes remote access possible using what is called a Reverse SSH tunnel. Because the embedded system is behind a firewall with no direct access from the Internet, the connection must be initiated from the embedded system. A reverse SSH tunnel is set up as follows:

the embedded systems make a SSH connection to an external server that is internet accessible.
The SSH connection is set up so that a port on the external server (let’s pick 2900) is forwarded back over the SSH tunnel to port 22 (port used for SSH) on the embedded system.
The developer connects to 2900 on the external server using SSH. Because port 2900 on the external server gets forwarded over the SSH tunnel to port 22 of the embedded system, the developer is actually logging into the embedded system.

This connection is illustrated in the figure below:

reverse_ssh_diagram

In the above figure, the external server is typically a system located in the developer’s office with only port 22 exposed to the Internet. Port 2900 is only accessible to developers on the same private network as the server.

Security

When talking about any type of remote access technology involving the internet, it is important to consider the security implications. The reverse SSH mechanism is fairly secure, as everything is encrypted. SSH is widely used for remote access to servers; therefore it receives a lot of testing and scrutiny. With the reverse SSH remote access setup, the most likely attack is probably to compromise the external server and then try to access the remote network the embedded system is located on through the embedded system. If the external server is properly secured, then this mechanism should be fairly safe. One of the advantages of this scheme is that the embedded system is not directly exposed to the Internet, therefore securing the embedded system itself is not all that important.

It is obviously very important that the people administering the remote network understand what you are doing — that is where this article can come in handy. If there are concerns about external developers accessing a network, one mode of operation may be for the the SSH tunnel to be manually started and stopped so that administrators of the remote network know exactly when remote access is enabled.

TIPS

A few ideas for making this work well:

It may be useful to put the reverse SSH connection in a cron job on the embedded system so that the connection is automatically set up. If you do this, I advise killing the connection and restarting every so often. Reverse SSH connections are somewhat fragile and occasionally need restarted if there is a network glitch.
Use the SSH TCPKeepAlive option. Some firewalls/routers will drop a network connection if there is no activity.

Summary

Using SSH for remote access is another example of the “Big Win” with Embedded Linux — how industrial grade technologies developed for use in server and desktop systems can be readily used in embedded systems (http://bec-systems.com/web/content/view/35/37/ ). Remote access during the deployment and testing of embedded systems can drastically reduce the amount of time and effort required to roll out an embedded system.