Search⌘ K
AI Features
Home
Courses
Concurrent Data Processing in Elixir

Choose the Right Dispatcher

Understand how to choose the appropriate dispatcher in GenStage for effective event routing within data processing pipelines. Explore the default DemandDispatcher, BroadcastDispatcher for sending events to all consumers, and PartitionDispatcher for partition-based event assignment. Learn to configure each dispatcher type to optimize concurrency and back-pressure management in Elixir pipelines.

We'll cover the following...

Introduction to dispatchers

There is one important component of GenStage we haven’t talked about yet—the dispatcher. When :producer and :producer_consumer stages send events to consumers, it is the dispatcher that sends the events. So far, we’ve used the default DemandDispatcher, but GenStage comes with two more. Let’s see what they do and how we can use them.

DemandDispatcher

We can specify the kind of dispatcher to use when we initialize the process using the :dispatcher key. The default is DemandDispatcher, which is equivalent to this configuration:

C++
def init(state) do
 {:producer, state, dispatcher: GenStage.DemandDispatcher}
end

The DemandDispatcher dispatcher sends events to consumers with the highest demand first. It is the dispatcher that we’ll use most often. However, there are cases where we may want to route events to consumers using a different strategy, where BroadcastDispatcher and PartitionDispatcher come ...