Design of Distributed Messaging Queue (Part 2)

In the previous lesson, we discussed the responsibilities of front-end servers and metadata services. In this lesson, we will focus on the main part of the design where queues and messages are stored, i.e., the backend service.

Backend service

This is the core part of the architecture where major activities take place. When the front-end receives a message, it refers to the metadata service to determine the host where the message has to be sent. The message is then forwarded to the host and is replicated on the relevant hosts to overcome the availability issue. The message replication in a cluster on different hosts can be performed by using one of the following two models.

1. Primary-secondary model
2. A cluster of independent hosts

Before going into the details of these models, let’s discuss the two types of cluster managers responsible for queue management; internal and external cluster managers. The differences between these two cluster managers are shown in the following table.