spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kriskalish <>
Subject Running Spark in Local Mode vs. Single Node Cluster
Date Mon, 22 Sep 2014 19:20:07 GMT
I'm in a situation where I'm running Spark streaming on a single machine
right now. The plan is to ultimately run it on a cluster, but for the next
couple months it will probably stay on one machine.

I tried to do some digging and I can't find any indication of whether it's
better to run spark as a single node cluster or just in local mode. As far
as I can tell, the only real difference is that it's difficult to configure
the executor memory in local mode.

I have been having problems with spark crashing in local mode so far, which
has lead me to do this research. I'll be migrating from Spark 1.0.2 to 1.1.0
in the next day or so to see if that helps.

Does anyone have any experience on the matter?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message