socat to Me: Connecting a Public and Private Subnet in AWS

Welcome to Create Impact, a new series from Aviture focused on the topics that inspire our engineers to innovate. In each article, an Aviture team member will take you on a deep-dive into a subject they’re passionate about, showing you the thinking behind cutting-edge engineering advances, the latest UX trends, development theories, and other unique topics that enable Aviturians to embrace the Art of the Possible for our clients.

In this post, Senior Software Engineer Mike Macaulay highlights a common issue that arises when working in Amazon Web Services (AWS), as well as how he and his team solved it creatively through a repeatable framework. He delves into the utility of this solution for anyone looking to fix a problem that could otherwise take hours and hours of programming time (not to mention potential security risks).

Keep reading to learn more!

Picture a community of IP addresses, and you have a simple way to think of the data and clients that make up Amazon Web Services (AWS).

These subnets can be public, meaning they have access to the wider Internet. Or they can be private, which is preferred, because if you have access to the outside internet, then hackers or security threats can come in from outside that specific subnet.

For many of our clients, having a public subnet would be a major problem. The Department of Defense’s Guardian program, for instance, requires pretty much the highest-security environment you could imagine. Remotely piloted aircrafts (RAFs) are responsible for critical missions domestically and abroad, and the data processed within the Guardian subnet needs to be protected from outside interference.

Hence, a private subnet is favored over a public subnet. You wouldn’t want any of that data to be in a public forum where anyone besides those with the right security clearance could access it. Because the subnet is private, the Guardian systems don't really have access to the internet in the traditional sense, but they still have to do some things that rely on the internet and the public subnets that populate it.

The Choice Is Yours

That leaves two options for connecting to the wider internet. One is called the NAT gateway in AWS. That's what your home router would typically use. According to AWS, “you can use a NAT gateway so that instances in a private subnet can connect to services outside your VPC but external services cannot initiate a connection with those instances.”

The other option is called an outbound proxy. An outbound proxy is like a gatekeeper that only allows certain accepted outside traffic to go through it. So it can lock down a subnet and restrict its access to certain sites, rather than having to rely on the creation of an entirely different port to enable the subnet to function. It's kind of like an extra layer of security for the whole system.

But here’s the catch: the subnet can’t automatically “talk” to the outbound proxy. They speak different languages. The outbound proxy only speaks and sends information via HTTPS, whereas our service can and does use multiple other ports.

That’s where tunneling comes in.

Tunnel Vision

With HTTP tunneling, you can establish a connection between your service and the outbound proxy that’s going to eventually communicate with the public subnet. Once you've established that connection, your service and the outbound proxy can talk binary back and forth, just straight-up ones and zeros. Once you're talking ones and zeros, it doesn't matter the language each communicates in — the connection allows them to communicate with each other.

To achieve this connection, we use a Linux utility program called socat (literal meaning: socket cat) that's available to all Unix and Linux systems. We could set this up on our host machine, but it’s far easier to set it up as a sidecar container to our service. A sidecar is a program that exists outside of yet is still attached to the main application, like a motorcycle sidecar.

For this example, it helps to think of socat as a universal translator. Our service lives on a private subnet and can only reach the wider internet via an outbound proxy, but our service wasn’t developed with that outbound proxy. Socat acts as our translator between the two.

Socat operates like a car traveling in a tunnel between two points. You leave your destination, and the tunnel is the path, but the car is what actually enables you to traverse the tunnel.

Protecting the Code

The reason this is such a valuable option is we don’t have to alter the code of our own service within the private subnet. Modifying code is a lot riskier because you have to go through a peer review process and you could break the original code. It also takes more time.

With socat, you don't have to go through that whole process and make it more complex. All you're really doing is creating the plumbing between the two things. This solution, and the thinking behind it, has helped us reframe not just this problem, but similar issues that we’ve encountered with other clients.

We’ve already used this framework in multiple places within the Guardian program and will likely continue to do so in Guardian and within different programs as well.

The savvy among you will probably think that, with this pattern, you effectively allow anything in your private subnet to talk to the whole world. Not so! Remember: Security first. Our sidecar container works on a per port basis, so we still make sure we only allow out connections we expect.

Security restrictions will only get tighter, and this is a useful tool in our toolbox for when a client needs to communicate with a public subnet but we don’t want to poke holes in the client’s security. Every hole is another attack vector, and this method allows us to avoid the holes entirely.

We can take the system we built with socat and apply DevOps strategy to iterate upon it. The problem was unique, but the pattern of using sidecar containers like socat as the go-between is not. We now have a solution for ensuring security remains paramount, the code doesn’t need to be altered, and our client’s private subnets are able to communicate with public subnets safely and effectively.