The blueprint of our job-processing system
Explore how to design a job-processing system in Elixir by creating a GenServer for each job, managing their state, implementing retries, and setting up a supervision tree for fault tolerance. This lesson helps you understand process initialization, state tracking, and concurrency patterns for improved performance.
We'll cover the following...
Blueprint of our job processing system
We used the Task module in the previous chapter to start a new process when sending each email. We’ll adopt the same approach for our job-processing system. Rather than having a single GenServer that does all the work, we start a GenServer process for each job. This helps us leverage concurrency and improve performance.
We also don’t want to lose any features that we have introduced so far in this chapter. We want to be able to retry failed jobs and configure the process to suit our needs. Finally, we’ll learn a new type of supervisor to help us manage these processes and provide fault tolerance. This figure gives us a high-level overview of what we’re going to build: