Concurrency vs. Parallelism in Distributed Systems

Modern applications are expected to handle thousands of user requests simultaneously while maintaining high performance.

To build systems that are both responsive and resilient, developers must understand the principles of concurrency and parallelism. Although these terms are often used interchangeably, they refer to distinct methods of executing tasks, each with unique implications for system architecture.

This lesson will deconstruct these two critical concepts, clarifying how they differ, the mechanisms that enable them, and when to apply each in real-world systems.

Introduction to concurrency and parallelism

At a high level, the distinction is about structure vs. execution.

Concurrency is about structuring a program to manage multiple tasks at once, making progress on them over a period of time. Parallelism is about executing multiple tasks simultaneously. The visual representation below illustrates this core difference in task structuring and execution.

One approach shows a single CPU core rapidly switching between tasks, making progress on both over time. The other shows multiple CPU cores, each dedicated to a task, allowing for true simultaneous execution.

Understanding this distinction is crucial for building modern distributed systems. It impacts how we write code, utilize hardware, and ultimately design for performance and scalability. Let’s explore how these concepts are implemented in software.

Concurrency in distributed systems

On a machine with a single CPU core, only one task can execute at any given moment. However, the system can rapidly switch between tasks, creating the illusion of simultaneous execution. This model is exceptionally efficient for I/O-bound applications.

To manage this, the operating system maintains the state of each task (or process). A task can be in one of several states, but the three primary ones are:

• Ready: The task is waiting and ready to execute as soon as the CPU becomes available.

• Running: The task is currently being executed by the CPU.

• Blocked: The task cannot run, even if the CPU is free, because it is waiting for an external event (like a database query to return or a file to be read).

The diagram below illustrates how an operating system manages concurrency by moving tasks between these states. This ensures that the CPU remains active (running a "Ready" task) even when other tasks are "Blocked" waiting for I/O.