System Design: The Pub-Sub Abstraction
Define the pub-sub abstraction, a core asynchronous communication pattern used in microservices and distributed systems. Understand how this pattern decouples producers and consumers, enabling greater scalability and independent development of system components.
We'll cover the following...
What is a pub-sub system?
Publish-subscribe (pub-sub) messaging is an asynchronous communication method popular in serverless and microservices architectures. It enables a system to send messages to multiple subsystems simultaneously. Services subscribed to a specific topic receive any message pushed to that topic.
For example, when a user posts on a social media platform, the platform delivers that post to their followers. In this scenario, the user acts as the publisher, the post is the message, and the followers are the subscribers.
Motivation
Distributed systems often rely on vast hardware infrastructure. Pub-sub systems enhance scalability by enabling asynchronous communication. This pattern decouples producers from consumers, allowing them to operate and scale independently. Consequently, engineers can add or remove components without disrupting the wider system.
How do we design a pub-sub system?
We divide the System Design process for a pub-sub system into the following lessons:
Introduction: Explore use cases, define requirements, and design the API.
Design: Siscuss two architectural approaches: one using message queues and the other using a broker.