Filesystem            Size  Used Avail Use% Mounted on

/dev/xvda1            7.9G  4.4G  3.5G  57% /

tmpfs                 7.4G  4.0K  7.4G   1% /dev/shm

/dev/xvdb              37G  3.3G   32G  10% /mnt

/dev/xvdf              37G  2.0G   34G   6% /mnt2

/dev/xvdv             500G   33M  500G   1% /vol

I'm on an EC2 cluster (c3.xlarge + 5 x m3) that I launched using the spark-ec2 scripts and a clone of spark from today. The job I am running closely resembles the collaborative filtering example. This issue happens with the 1M version as well as the 10 million rating version of the MovieLens dataset.

I have seen previous questions, but they haven't helped yet. For example, I tried setting the Spark tmp directory to the EBS volume at /vol/, both by editing the spark conf file (and copy-dir'ing it to the slaves) as well as through the SparkConf. Yet I still get the above error. Here is my current Spark config below. Note that I'm launching via ~/spark/bin/spark-submit.

conf = SparkConf()

conf.setAppName("RecommendALS").set("spark.local.dir", "/vol/").set("spark.executor.memory", "7g").set("spark.akka.frameSize", "100").setExecutorEnv("SPARK_JAVA_OPTS", " -Dspark.akka.frameSize=100")

sc = SparkContext(conf=conf)