Design Considerations in Distributed Messaging Queue

Before embarking on the design of a distributed messaging queue, let’s discuss some major factors that could significantly affect the design. These include the order of messages, ordering effect on performance, and managing concurrent access to the queue. Each of these factors is discussed below.

Messages ordering

A messaging queue is used to receive messages from producers which are consumed by the consumers at their own pace. Some operations are critical which require strict ordering of the execution of the tasks driven by the messages in the queue. For example, while chatting over a messenger with a friend, the messages should be delivered in order; otherwise, such communication can be confusing, to say the least, or worse (what will happen if we are sharing a recipe with our friend in multiple messages and some of them get re-ordered before the reception!). Similarly, emails received by a user from different users may not require strict ordering. Therefore, in some cases, the strict order of incoming messages in the queue is essential, while many use-cases can tolerate some re-ordering.

Let’s discuss the following two categories of messages ordering in a queue:

• Best-effort ordering
• Strict ordering