Spark applications run as separate processes in distributed clusters that, once connected, are coordinated by the SparkContext in its controller program. Spark acquires executors on cluster nodes that perform the calculations.
To run an application on the Spark cluster, pass the
of the master as to the SparkContext constructor.
A shell can be run against the cluster by typing,
./bin/spark-shell --master spark://IP:PORT.
--total-executor-cores <number of cores> can be set to control the number of cores the spark-shell uses.
View all Courses