A Short History of Infrastructure Management

The beginning

A long time ago in a galaxy far, far away…

We would order servers and wait for months until they arrived. To make our misery worse, even after they came, we’d wait for weeks (sometimes even months) until they were placed in racks and provisioned.

At that time, only a select few people could access these servers. If someone did something that should not have been done, we would face an extended downtime. On top of that, nobody knew what was running on those servers.

Manual provisioning and installations were a nightmare because even after putting a lot of effort into documentation, given enough time, the state of the servers would always diverge from the documentation. Sysadmins were the key people who could handle these servers.

Configuration management and relevant tools

Managing the configuration means tracking and controlling changes in the software. Configuration management tools enable us to determine what was changed, who changed it and much more.

Then came configuration management tools like CFEngine.

Pros

CFEngine was based on promise theory and was capable of putting a server into the desired state no matter what its actual state was.

It allowed us to specify the state of static infrastructure and have a reasonable guarantee that it will be achieved.

Another big advantage it provided is the ability to have, more or less, the same setup for different environments. Servers dedicated to testing could be (almost) the same as those assigned to production.

Cons

Unfortunately, usage of CFEngine and similar tools was not yet widespread. We had to wait for virtual machines before automated configuration management became a norm. However, CFEngine was not designed for virtual machines. it was meant to work with static, bare metal servers. Still, CFEngine was a massive contribution to the industry even though it failed to get widespread adoption.

After CFEngine, other similar tools like Chef, Puppet, Ansible, Salt, and more were introduced. We’ll discuss these tools later on. For now, let’s discuss the next evolutionary improvement.

Besides forcing us to be patient, physical servers were a massive waste of resource utilization. They came in predefined sizes and, since waiting time was considerable, we often opted for big ones. The bigger, the better. That meant that an application or a service usually required less CPU and memory than the server offered. Unless you do not care about costs, that meant that we’d deploy multiple applications to a single server. The result was a dependencies nightmare. We had to choose between freedom and standardization.

Freedom meant that different applications could use different runtime dependencies while standardization involved systems architects deciding the only right way to develop and deploy something.

Virtual machines

Then came the virtual machines (VMs). Which were a massive improvement over bare metal infrastructure.

• They allowed us to be more precise with hardware requirements.

• They could be created and destroyed quickly.

• They could differ, i.e., a single physical server could have multiple VMs running in isolation–one VM could host a Java application, and the other could be dedicated to Ruby on Rails.

• We could get them in a matter of minutes instead of waiting for months–still, it took quite a while to complete the task.

Even though the advantages brought by VMs were numerous, years passed until they were widely adopted. Even then, the adoption was usually wrong. Companies often moved the same practices used with bare metal servers into virtual machines. We could have identical servers in different environments. Companies started copying VMs. While that was much better than before, it did not solve the problem of missing documentation and the ability to create VMs from scratch. Still, multiple identical environments are better than one, even if we don’t know what’s inside.

Mutability vs. immutability

The configuration management tools helped spread the adoption of “infrastructure as code” principles. But the problem was they were designed with static infrastructure in mind. On the other hand, VMs opened the doors to dynamic infrastructure, in which VMs are continuously created and destroyed. Mutability and constant creation and destruction clashed.

Mutable infrastructure is well suited for static infrastructure. It does not respond well to challenges brought by the dynamic nature of modern data centers. Mutability (changeable at runtime) had to give way to immutability (nothing can be tweaked at runtime).

When ideas behind immutable infrastructure started getting traction, people began combining them with the concepts behind configuration management. However, the tools available at that time were not fit for the job. They (Chef, Puppet, Ansible, and the like) were designed with the idea that servers are brought into the desired state at runtime. On the other hand, immutable processes assume that almost nothing is changeable at runtime. Artifacts were supposed to be created as immutable images. In the case of infrastructure, VMs were created from images, and not changed at runtime. If an upgrade was needed, a new image would be created, followed by a replacement of old VMs with new ones based on the new image. Such processes brought speed and reliability. With proper tests in place, immutable was found to be more reliable than mutable.

Subsequently, we got tools capable of building VM images. Today, they are ruled by Packer. Configuration management tools quickly jumped on board, claiming to work equally well for configuring images as servers, at runtime. However, that was not the case due to the logic behind those tools. They were designed to put a server that is in an unknown state into the desired state, assuming the current state is unknown. VM images, on the other hand, are always based on an image with a known state. If, for example, we choose Ubuntu as a base image, we know what’s inside it.

Adding additional packages and configurations is easy. There is no need for things like “if this, then that, otherwise something else.” A simple shell script is as good as any configuration management tool when the current state is known. Creating a VM image is reasonably straightforward with Packer alone. Still, not all was lost for configuration management tools. We could still use them to orchestrate the creation of VMs based on images and potentially do some runtime configuration that couldn’t be baked in.

Cloud hosting

The way we orchestrated infrastructure had to change as well. A higher level of dynamism and elasticity was required. That became especially evident with the emergence of cloud hosting providers like Amazon Web Services (AWS) and, later on, Azure and Google Computer Engine (GCE).

They showed us what could be done. While some companies embraced the cloud, others went into defensive positions. Some companies had excuses like “We can build an internal cloud,” “AWS is too expensive,” “We would, but we can’t because of legislation,” etc., in an effort to maintain the status quo. That is not to say that there was no truth in those statements but that, more often than not, they were used as excuses for real reasons.

Still, the cloud did manage to become the preferred way to do things, and companies moved their infrastructure to one of the cloud hosting providers. Or, at least, started thinking about it. The number of companies that have abandoned on-premise infrastructure is continuously increasing, and we can safely predict that the trend will continue.

Still, the question remains, how do we manage infrastructure in the cloud with all the benefits it gives us? How do we handle its highly dynamic nature? The answer has arrived in the form of vendor-specific tools like CloudFormation or agnostic solutions like Terraform. When combined with tools that allow us to create images, they represent a new generation of configuration management. What we are talking about here is full automation backed by immutability.

Modern infrastructure

We’re living in an era without the need to SSH into servers.