Task Distribution Patterns

Although delegating costly tasks to multiple local processes is an effective approach, it can’t be scaled beyond the boundaries of a single machine, so in this section, we’re going to see how it’s possible to use a similar pattern in a distributed architecture, using remote workers located anywhere in a network.

The idea is to have a messaging pattern that allows us to spread tasks across multiple machines. These tasks might be individual chunks of work or pieces of a bigger task split using a divide-and-conquer approach.

If we look at the logical architecture represented in the following illustration, we should be able to recognize a familiar pattern: