spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shahab <>
Subject [Spark on Amazon EMR] : File does not exist: hdfs://ip-x-x-x-x:/.../spark-assembly-1.4.1-hadoop2.6.0-amzn-0.jar
Date Wed, 09 Sep 2015 14:28:20 GMT
I am using Spark on Amazon EMR. So far I have not succeeded to submit the
application successfully, not sure what's problem. In the log file I see
the followings. File does not exist:

However, even putting spark-assembly-1.4.1-hadoop2.6.0-amzn-0.jar in the
fat jar file didn't solve the problem. I am out of clue now.
I want to submit a spark application, using aws web console, as a step. I
submit the application as : spark-submit --deploy-mode cluster --class
mypack.MyMainClass --master yarn-cluster s3://mybucket/MySparkApp.jar Is
there any one who has similar problem with EMR?


View raw message