Re: The difference between driver and master Spark

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: The difference between driver and master Spark

andy
Administrator
The master is what monitors and manages jobs. It can be a master in a Spark
standalone scheduler (which itself might be running inside of YARN) or a
Mesos master.

The driver is the program that the user interfaces with to talk to their
Spark job. That communication is primarily dive through a SparkContext
object.

Have you read through http://spark.apache.org/docs/latest/index.html yet?
If not, it and the other docs might be helpful.

In the future, this level of question would probably be better suited for
the user list.

Best,
Andy
On Mar 31, 2014 7:48 AM, "Dan" <[hidden email]> wrote:

Hi,

I've been recently reading spark code and confused about driver and
master. What's the difference between them?

When I run spark in standalone cluster, from the log it seems that the
driver has not been launched.

Thanks,
Dan




--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/The-difference-between-driver-and-master-in-Spark-tp6158.html
Sent from the Apache Spark Developers List mailing list archive at
Nabble.com.
Loading...