Execution of a Spark Application

This lesson explains the role of the DAG and Task schedulers.

We'll cover the following

Execution of a Spark Application

As discussed earlier, a Spark job is initiated when an action is performed. Internally, the SparkContext object’s runJob(...) method is invoked. The call is passed on the scheduler. The scheduler runs as part of the driver and has two parts:

  • DAG Scheduler
  • Task Scheduler

Get hands-on with 1200+ tech skills courses.