Virtual Machines in the Cloud

Traditional web services

At the time of writing, Amazon Web Services is far and away the dominant cloud platform. Google Cloud is gaining traction thanks to an attractive pricing model, but it has a long way to go before its workload approaches AWS. The world can change pretty quickly, though. While advanced cloud features definitely help with lock-in, compute, and storage capacity is more fungible.

It’s evident now that traditional applications can run in the cloud. No matter what we say about “lift and shift” efforts, they do run. Despite that, a cloud native system will have better operational characteristics, especially in terms of availability and cost.

Virtual machine operations

Any individual virtual machine in the cloud has worse availability than any individual physical machine (assuming equally skilled data center engineering and operations). If we think about it in terms of “moving parts,” we’ll see why that has to be the case. A virtual machine in the cloud runs atop a physical host, but with an extra operating system in the middle. It can be started or stopped without notice by the management APIs (in other words, the control plane software). It also shares the physical host with other virtual machines and may contend for resources. If we’ve been running in AWS for any length of time, we’ll have encountered virtual machines that were killed for no apparent reason. If we have long-running virtual machines, we may even have gotten a notice from AWS informing us that the machine has to be restarted.

IP/ID addresses

Another factor that presents a challenge to traditional applications is the ephemeral nature of machine identity. A machine ID and its IP address are only there as long as the machine keeps running. Most traditional application configurations keep hostnames or IP addresses in config files. But in AWS, a VM’s IP address changes on every boot. If the application needs to keep those addresses in files, then we have to rent Elastic IP addresses from Amazon. That works well enough until we need a lot of them. A basic AWS account has a limit on how many addresses it can procure.

The general rule is that VMs have to “volunteer” to do work, rather than having a controller dole the work out. That means a new VM should be able to start up and join whatever pool of workers handles load. For HTTP requests, autoscaling and load balancers (either elastic load balancers or application load balancers) are the way to go. For asynchronous load, use competing consumers on a queue.

NIC connections

When it comes to network interfaces on those cloud VMs, the default is pretty simple: one NIC with a private IP address. This isn’t always what we want, though. There’s a limit to how much traffic a single NIC can support, based on the number of sockets available. Socket numbers only range from 1 to 65,535 so at best a single NIC can support about 64,000 connections. We may want to set up more production NICs just to handle more simultaneous connections.

Another good reason to set up another NIC is for monitoring and management traffic. In particular, it’s a bad idea to have SSH ports available on front-end NICs for every server. It’s better to set up a single entry point, known as a bastion or jumphost server, with strong logging on SSH connections and then use the private network to get from there to other VMs. Networking these VMs together presents its own set of challenges and solutions.