spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From juanpedromoreno <>
Subject Problem with Spark on Mesos
Date Tue, 04 Mar 2014 12:20:12 GMT
Hi, I'm using vagrant. I've built a cluster with MESOS and SPARK with the
following structure:

- 2 zookeeper nodes
- 2 master nodes  
- 3 slave nodes

In each master node, I've installed MESOS and Spark. 
My $SPARK_HOME/conf/ contains:

export MESOS_NATIVE_LIBRARY=/usr/local/lib/
export MASTER=zk://,

The issue I'm having is that executors, launched by the slaves nodes, always
download the spark distribution from 
Disk usage of slave nodes is continuously increasing, in addition to the
introduced latency which is due to downloading and decompressing the tgz
file. Is there any way to avoid it?

Thanks in advance.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message