Essential Network Protocols for Distributed Systems

A distributed system’s greatest strength and weakness are that its components are separate. Without a robust communication backbone, it’s just a collection of isolated computers. The choice of how these components talk to each other is a critical decision in System Design, influencing a system’s performance, reliability, and scalability.

This lesson will break down the essential network protocols that enable this communication. Let’s start with a simple model.

The diagram above shows a simple end-to-end flow. A client request travels to a server, and different protocols govern this communication. We will now explore these protocols, starting with the application layer, where most engineers interact with them day-to-day.

HTTP as an architectural backbone

The Hypertext Transfer Protocol (HTTP) is the foundation of communication on the web. As an application-layer protocol, it defines how clients and servers exchange messages. For System Design, knowing how HTTP works is essential for building APIs and web services.

The stateless nature of HTTP

The most important characteristic of HTTP is that it is statelessA communication protocol where the server does not store any information about previous client requests. Each request is treated as a new, independent transaction.. Each request from a client to a server must contain all the information needed to understand and complete the request. The server does not retain any memory of past requests. This design simplifies server logic and enhances scalability, but it means stateful information (like user sessions) must be managed externally.

Core components of an HTTP request

An HTTP request sent by a client is made up of the following parts:

• Methods: Verbs that define the desired action, for example, GET is used to retrieve data, POST to submit data, PUT to replace a resource, PATCH to apply a partial update, and DELETE to delete a record.

• Headers: Key-value pairs that carry metadata about the request, such as authentication tokens or the format of the data being sent.

• Body: The actual data being sent to the server, such as a JSON object in a POST request.

Core components of an HTTP response

An HTTP response sent by the server includes:

• Status code: A three-digit code that indicates the outcome of the request (e.g., 200 OK, 404 Not Found, 500 Internal Server Error).

• Headers: Key-value pairs that carry metadata about the response, such as caching instructions or the content type.

• Body: The actual content being returned. Note that some responses (like a HEAD request or a 204 No Content status) intentionally have no body.

The diagram below shows how a client makes a simple GET request and how the server responds.

A sample HTTP request and response

Here is what a complete HTTP request and response look like in practice:

After understanding HTTP’s nature, core components, and request and response format, let's analyze its evolution and the features each update offers.

The evolution of HTTP

HTTP has gone through several versions, each improving performance and scalability:

Each version addressed limitations of its predecessor, making HTTP more efficient and resilient for distributed systems.

HTTP provides the “what” of web communication, but it runs on top of transport protocols that determine how data moves across the network. That’s where TCP and UDP come in.

Comparing TCP and UDP for distributed systems

At the transport layer, engineers often face a trade-off between reliability and speed. Two core protocols embody this trade-off: Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). The following diagram illustrates the fundamental difference in their communication styles.

As the diagram shows, TCP establishes a formal connection and confirms data delivery, while UDP sends data without any guarantees. Let’s examine TCP’s reliability.

The reliability of TCP

TCP is connection-oriented, meaning it sets up a connection before any data is exchanged. Think of it like making a phone call; you wait for the other person to pick up before speaking. This setup is called a three-way handshakeThe process TCP uses to establish a connection between a client and a server. It involves three steps: SYN (synchronize), SYN-ACK (synchronize-acknowledgment), and ACK (acknowledgment)..

Once connected, TCP provides two key guarantees:

• Reliable delivery: It uses acknowledgments (ACKs) to confirm that data packets were received. If an ACK isn’t received, the packet is re-sent.

• Ordered delivery: It uses sequence numbers to ensure that packetsSmall segments of a larger message. When data is sent over a network, it is broken down into packets. are reassembled in the correct order at the destination.

These guarantees make TCP reliable but slower, because of the overhead from handshakes, acknowledgments, and ordering. It’s the default for scenarios where correctness is non-negotiable, like web browsing (HTTPS), file transfers (FTP), and email (SMTP).

The speed of UDP

UDP is connectionless, a fire-and-forget protocol. It just sends packets (datagrams) without waiting for setup or acknowledgment. Imagine sending a postcard; you drop it in the box, but there’s no guarantee it arrives.

UDP prioritizes speed and low latency over reliability. It has no mechanism for guaranteed or ordered delivery. This makes it suitable for applications where speed is more important than perfect data integrity. Good examples include:

• Video and voice streaming: Losing a single frame or a millisecond of audio is often better than pausing the entire stream to wait for a retransmission.

• Online gaming: Real-time player movements need to be transmitted with minimal delay.

• DNS lookups: A simple query-response that primarily uses UDP for speed, but can fall back to TCP for larger responses. If a UDP request is lost, the client simply retries.

Making the right trade-off between TCP and UDP is a classic System Design problem. To help clarify this choice, the following table summarizes the key differences between them.

Now that we’ve covered the key application (HTTP) and transport (TCP/UDP) protocols, let’s place them within the practical TCP/IP framework.

TCP/IP model

The seven-layer OSI Model is often taught as the theoretical standard for networking. But in practice, most real-world systems, including the internet, use the simpler TCP/IP model.

The TCP/IP model has four layers, but in System Design, you’ll primarily work with the top three:

• Application layer: This is where user-facing protocols live. It's where you’ll make decisions about HTTP, gRPChttps://grpc.io/, APIs, and data formats like JSON.

• Transport layer: This layer handles end-to-end communication. As we’ve discussed, you choose TCP (for reliability) or UDP (for speed) here.

• Network layer: This layer handles addressing and routing packets across networks using the Internet Protocol (IP).

• Link layer: This is the physical foundation (e.g., Ethernet, Wi-Fi) that the other layers are built on, but it’s rarely a direct concern in application design.

The following diagram shows how these layers work together in a real-world flow.

Understanding these layers is invaluable for troubleshooting. When a service is unreachable, is it an application error (application layer), a blocked port (transport layer), or a routing problem (internet layer)? This layered thinking helps narrow down the possibilities.

The following table summarizes the TCP/IP layers and their relevance to System Design.

If you’re not sure how to do this, click the “Want to know the correct answer?” button.

Conclusion

In this lesson, we saw how HTTP enables stateless communication at the application layer, while TCP and UDP define the transport trade-offs between reliability and speed.

Finally, we placed these protocols within the TCP/IP model to understand how they interact in real-world systems. These networking choices form the foundation of scalable, resilient System Design. In the next lesson, we’ll build on this by exploring synchronous vs. asynchronous communication, two different interaction patterns that determine when systems exchange information and how they balance latency, scalability, and reliability.