spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Clasen <scott.cla...@gmail.com>
Subject Re: Spark Streaming + Kafka + Mesos/Marathon strangeness
Date Thu, 27 Mar 2014 17:32:17 GMT
I think now that this is because spark.local.dir is defaulting to /tmp, and
since the tasks are not running on the same machine, the file is not found
when the second task takes over.

How do you set spark.local.dir appropriately when running on mesos?



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Kafka-Mesos-Marathon-strangeness-tp3285p3356.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message