Horizontal Scaling vs. Vertical Scaling and Multi-Tenancy

In this section, we will learn various cloud concepts:

1. Scaling on the cloud
2. Degradation of services
3. Availability Vs durability on the cloud
4. Single and multi-tenancy applications
5. Types of cloud deployments

Scaling out (horizontal scaling) vs. scaling up (vertical scaling)

Horizontal scaling means that you scale by adding more machines to your pool of resources. For example, as traffic increases, you add more web servers to handle it.

Vertical scaling means that you scale by adding more power (CPU, RAM) to an existing machine.

On the cloud, you scale horizontally!

Graceful Degradation of Services

Applications need to handle excessive load and failure gracefully, whether it’s the application or a dependent service under load.

The Site Reliability Engineering handbook describes graceful degradation in applications as offering “responses that are not as accurate as or that contain less data than normal responses, but that are easier to compute” when under excessive load.

Modern applications are built with breakers that prevent other microservices from being overloaded if a microservice malfunctions. They also need sufficient telemetry to support troubleshooting and identify what is causing the problem.

If you take a moment to think about all the apps on your phone that you use, when was the last time any of them had an outage? The probability of an outage is so low because the system design and architecture of modern applications account for every single point of failure, from the application to the database, the load balancer, and even the infrastructure.

Graceful failure is about making sure that in the event of failure at any level that there is a backup that takes the responsibility of the component that failed, and in the end, there is no perceived customer impact. Note the word ‘perceived’. Let’s take, for example, that a large eCommerce/retail website calculates the tax for each shipment before you place the order. If the tax microservice goes down, then maybe you do not charge the customer tax for that one transaction; the cost will be covered by the organization.

Availability vs. durability on the cloud

Availability

Availability and durability are two very different aspects of data accessibility. Most cloud services offer a 99.9999% availability. This varies by provider and by each service that is offered by the provider.

Availability refers to system uptime, i.e., the storage system is operational and can deliver data upon request. Historically, this has been achieved through hardware redundancy, so that if any component fails, access to data remains available.

Durability

Durability, on the other hand, refers to long-term data protection, i.e., the stored data does not suffer from bit rot, degradation, or other corruption. Rather than focusing on hardware redundancy, it focuses on data redundancy to ensure data is never lost or compromised.

Availability and durability serve different objectives. For data centers, availability/uptime is a key metric for operations, as any minute of downtime is costly. The measurement focuses on storage system availability. But what happens when a component, system, or even the data center goes down? Will your data be intact when the fault is corrected?

This illustrates the equal importance of data durability. When an availability fault is corrected, it is essential that access to uncorrupted data is restored. With the explosion of data, the potential for mining, and the growing need for longer retention rates (for everything), you can imagine how this is paramount for business success.

Consider the potential competitive, financial, or legal impact of being unable to retrieve the archived master/reference copy of the data. Hence, both data availability and data durability are essential for short- and long-term business success.

Single- vs. multi-tenancy cloud deployment

Single-tenancy or multi-tenancy is in the context of SaaS applications. A SaaS application can choose to be one of the other. Most modern applications choose to be multi-tenant applications.

Multi-tenancy

Multi-tenancy means that a single instance of the software and its supporting infrastructure serves multiple end customers. Each customer shares the software application and also shares a single database. The data is tagged in the database as belonging to one customer or another, and the software is smart enough to know which customer it belongs to.

Single-tenancy

In a single-tenancy approach, a single instance of the software and all of its supporting infrastructure serves a single customer. With a single tenancy, each customer has their own independent database and instance of the application software as well. With this option, there’s essentially no sharing. Everyone has their own, separate from everyone else.

Think about a SaaS application serving customers like Coke and Pepsi; sometimes these companies care that they are not sharing the database tier, so there is no possibility of information leaking. If this were a concern, they would go with a single-tenancy rather than a multi-tenancy approach.

With single-tenant architectures, the marginal cost of adding new customers never goes down.

Maintenance

With single-tenant, maintenance is incremental. If you botch it, you typically only take down a single team. Instead of deploying a single app update, you’re deploying N app updates. Instead of migrating one database, you’re migrating N databases. On the surface level, it sounds like this would create more work for you, but since the systems are isolated and identical, most of that work can be automated. All you need is a bit of tooling to orchestrate the updates.

Resilience

Single-tenancy creates resilience at the system level. Outside of DNS-level DDoS attacks, there are very few ways to take down the entire system. A team may experience problems, but it’s unlikely those problems will extend beyond that single instance.

Types of cloud deployments

Hybrid cloud

A hybrid cloud is an environment that uses a mix of on-premises, private cloud, and/or third-party public cloud services, with orchestration between the two platforms.

Multi-cloud

A multi-cloud strategy is the use of two or more cloud services. While a multi-cloud deployment can refer to any implementation of multiple software as a service (SaaS) or platform as a service (PaaS) cloud offerings, today, it generally refers to a mix of public infrastructure as a service (IaaS) environments, such as Amazon Web Services, Microsoft Azure, and Oracle Cloud Infrastructure.

Multi-cloud was and still is seen as a way to prevent data loss or downtime due to a localized component failure in the cloud. The ability to avoid vendor lock-in was also an early driver of multi-cloud adoption. A multi-cloud strategy also offers the ability to select different cloud services or features from different providers.

The pros and cons of a multi-cloud strategy

There are several commonly cited advantages of multi-cloud computing, such as avoiding vendor lock-in, finding the optimal cloud service for a particular business or technical need, and increased redundancy.

However, there are some potential drawbacks. For example, most public cloud providers offer volume discounts, where prices are reduced as customers buy more of a particular service. It becomes more difficult for an organization to qualify for those discounts when it doesn’t concentrate its business on a single cloud provider.

In addition, a multi-cloud deployment requires IT staff with expertise across multiple cloud platforms and providers. Workload or application management in multi-cloud computing can also be challenging, as data moves between cloud platforms.

Multi-cloud computing vs. hybrid cloud computing

Multi-cloud and hybrid cloud computing are similar, but different IT infrastructure models. In general, the hybrid cloud refers to a cloud computing environment that uses a mix of an on-premises private cloud and a third-party public cloud, with orchestration between the two. An enterprise often adopts a hybrid cloud to achieve a specific task, such as the ability to run workloads in-house, and then burst into the public cloud when computing demands spike.

However, multi-cloud doesn’t preclude a hybrid cloud, and a hybrid cloud could be part of the multi-cloud deployment. The two models are not an either-or; it depends on what a business hopes to achieve.

The basic benefits of multi-tenant

Resilience

Both architectures offer their own form of resilience. Multi-tenant creates resilience at the team level. Each team is serviced by multiple instances, spread across multiple regions/zones, and hosted behind load balancers. A team is unlikely to experience issues unless there is a system-wide outage.

Maintenance

In multi-tenant environments, deploys are typically all-or-nothing. Maintenance of multi-tenant systems can be scary. You push out a single update, and every customer is immediately on the new system. If you botch it, you take down the entire system.

Easy upgrades

Instead of a vendor needing to update every instance of their software across a large number of servers, they can update a single, central application or codebase, with the changes instantly available to all users. With a multi-tenant application, spinning up a new cloud and application for a new customer is incredibly easy and can be done very quickly.

Easy customizations

With multi-tenancy-based applications, you can provide an additional layer for customization while maintaining an underlying codebase that remains constant for all users, including, of course, new customers.

Ongoing cost savings

Multi-tenancy speeds up upgrades, saves time (and also cost), but in addition, the server/cloud requirements for a multi-tenancy application cost much less. The opportunity to save money takes many forms and becomes greater as the application scales up. This reduces the cost of doing business for the vendor, and savings can be passed on to customers.