Scale Up a Stage with Extra Processes

Add Registry as the first child

Since we are adding more processes of the same type, let’s create a Registry and keep track of them. We can add it as the first child process in our main supervision tree in application.ex:

#file path -> scraper/lib/scraper/application.ex
children = [
  {Registry, keys: :unique, name: ProducerConsumerRegistry}, 
  PageProducer,
  OnlinePageProducerConsumer,
  PageConsumerSupervisor
]

We call the new process ProducerConsumerRegistry to keep its name short, but we can also call it OnlinePageProducerConsumerRegistry if we’d like to be specific.

Add another producer-consumer

Now, we want to add another OnlinePageProducerConsumer, but we must assign it a unique name and an ID before adding it to the main supervision tree. We may want to add more of these processes in the future, so let’s create a helper function to avoid repetitive code: