spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shushant Arora <shushantaror...@gmail.com>
Subject spark --files permission error
Date Mon, 03 Aug 2015 08:34:05 GMT
Is there any setting to allow --files to copy jar from driver to executor
nodes.

When I am passing some jar files using --files to executors and adding them
in class path of executor it throws exception of File not found

15/08/03 07:59:50 WARN TaskSetManager: Lost task 8.0 in stage 0.0 (TID 8,
ip): java.io.FileNotFoundException: ./jar (Permission denied)
        at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
        at
org.spark-project.guava.common.io.Files$FileByteSink.openStream(Files.java:223)


Running program as :
spark-submit --class classname  --files "externaljarname.jar"
--driver-class-path "externaljarname.jar" --conf
"spark.executor.extraClassPath=externaljarname.jar" mainjar.jar

Mime
View raw message