Concurrent Data Processing in Elixir/

...

/

Limit Concurrency and Inspect the Supervisors

Limit Concurrency and Inspect the Supervisors

Learn to limit the number of jobs we can run concurrently and examine the count_children function of their supervisor.

We'll cover the following...

Limit the concurrency of import jobs

We now put our new running_imports/0 to use. Let’s add this check to Jobber.start_job/1:

C++
#file path -> jobber/lib/jobber.ex
def start_job(args) do
  if Enum.count(running_imports()) >= 5 do
    {:error, :import_quota_reached}
  else
    DynamicSupervisor.start_child(JobRunner, {JobSupervisor, args})
  end
end

Try starting more than five jobs. After the fifth attempt, we should see this error result: {:error, :import_quota_reached}.

This is the executable command:

C++
Jobber.start_job(work: good_job, type: "import")

Here’s the output we get:

iex(1)> Jobber.start_job(work: good_job, type: "import")
{:ok, #PID<0.179.0>}
iex(2)> Jobber.start_job(work: good_job, type: "import")
{:ok, #PID<0.183.0>}
iex(3)> Jobber.start_job(work: good_job, type: "import")
{:ok, #PID<0.186.0>}
iex(4)> Jobber.start_job(work: good_job, type: "import")
{:ok, #PID<0.189.0>}
iex(5)> Jobber.start_job(work: good_job, type: "import")
{:ok, #PID<0.192.0>}
iex(6)> Jobber.start_job(work: good_job, type:
...