Design of a Distributed Task Scheduler

Explore and connect the design components of the distributed task scheduler.

Let’s identify the components used in this design:

Components

We can consider scheduling at many levels. We could be asked to design scheduling that is done internally by an organization to run tasks on their own cluster of machines. There, they have to find ample resources and need to decide which task to run first.

On the other hand, we could also be asked to design scheduling that a cloud provider uses to schedule tasks coming from multiple clients. Cloud providers need to decide which task to run first and which clients to handle first to provide appropriate isolation between different tenants.

So, in general, the big components of our system are:

  • Clients: They initiate the task execution.
  • Resources: The task is executed on these components.
  • Scheduler: A scheduler performs processes between clients and resources and decides which task should get resources first.
Press + to interact
Scheduler putting tasks into a queue for resource allocation
Scheduler putting tasks into a queue for resource allocation

As shown in the above illustration, it is necessary to put the incoming tasks into a queue. It is because of the following reasons:

  • We might not have sufficient resources available right now.
  • There is task dependency, and some tasks need to wait for others.
  • We need to decouple the clients from the task execution so that they can hand off work to our system. Our system then queues it for execution.

Let’s design a task scheduling system that should be able to schedule any task. Often, many tasks are relatively short-lived—from seconds to minutes. For long-running tasks, we might need the ability of periodic checksumming and restoration at the application level to recover from possible failures.

Let’s assume that some single server in our fleet can meet the computational needs of each task. For tasks that need many servers, either the ...

Level up your interview prep. Join Educative to access 70+ hands-on prep courses.