Cloning and Load Balancing

Traditional, multithreaded web servers are usually only scaled horizontally when the resources assigned to a machine can’t be upgraded anymore, or when doing so would involve a higher cost than simply launching another machine.

By using multiple threads, traditional web servers can take advantage of all the processing power of a server, using all the available processors and memory. Conversely, Node.js applications, being single-threaded, are usually scaled much sooner compared to traditional web servers. Even in the context of a single machine, we need to find ways to “scale” an application in order to take advantage of all the available resources.

Note: In Node.js, vertical scaling (adding more resources to a single machine) and horizontal scaling (adding more machines to the infrastructure) are almost equivalent concepts: both, in fact, involve similar techniques to leverage all the available processing power.

Don’t be fooled into thinking about this as a disadvantage. On the contrary, being almost forced to scale has beneficial effects on other attributes of an application, in particular, availability and fault tolerance. In fact, scaling a Node.js application by cloning is relatively simple and it’s often implemented even if there’s no need to harvest more resources, just for the purpose of having a redundant, fail-tolerant setup.

This also pushes the developers to take into account scalability from the early stages of an application, making sure the application doesn’t rely on any resource that can’t be shared across multiple processes or machines. In fact, an absolute prerequisite to scaling an application is that each instance doesn’t have to store common information on resources that can’t be shared, such as memory or disk. For example, in a web server, storing the session data in memory or on disk is a practice that doesn’t work well with scaling. Instead, using a shared database will ensure that each instance will have access to the same session information, wherever it’s deployed.

Let’s now introduce the most basic mechanism for scaling Node.js applications: the cluster module.

The cluster module

In Node.js, the simplest pattern to distribute the load of an application across different instances running on a single machine is by using the cluster module, which is part of the core libraries. The cluster module simplifies the forking of new instances of the same application and automatically distributes incoming connections across them, as shown in the illustration below.