Re: The difference between driver and master Spark

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Re: The difference between driver and master Spark

The master is what monitors and manages jobs. It can be a master in a Spark
standalone scheduler (which itself might be running inside of YARN) or a
Mesos master.

The driver is the program that the user interfaces with to talk to their
Spark job. That communication is primarily dive through a SparkContext

Have you read through yet?
If not, it and the other docs might be helpful.

In the future, this level of question would probably be better suited for
the user list.

On Mar 31, 2014 7:48 AM, "Dan" <[hidden email]> wrote:


I've been recently reading spark code and confused about driver and
master. What's the difference between them?

When I run spark in standalone cluster, from the log it seems that the
driver has not been launched.


View this message in context:
Sent from the Apache Spark Developers List mailing list archive at