spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mr rty ff <>
Subject Spark submit running spark-sql-perf and additional jar
Date Sun, 23 Oct 2016 20:20:46 GMT
I run the following script

home/spark-2.0.1-bin-hadoop2.7/bin/spark-submit --conf  "someconf" "--jars /home/user/workspace/auxdriver/target/auxdriver.jar,/media/sf_VboxShared/tpc-ds/spark-sql-perf-v.0.2.4/spark-sql-perf-assembly-0.2.4.jar
--benchmark DatabasePerformance --iterations 1 --sparkMaster somemaster --location "location"
--scaleFactor 50 --appName "nvm"
I get the following error
Error: Cannot load main class from JAR file:/home/irina/workspace/stocator-s3/target/stocator-s3-1.0-jar-with-dependencies.jar,/media/sf_VboxShared/tpc-ds/spark-sql-perf-v.0.2.4/spark-sql-perf-assembly-0.2.4.jar
If I change the order I get
Error: Unknown argument '/home/user/workspace/auxdriver/target/auxdriver.jar'

If I run without auxdriver it works ok but I need it to run it with it to test the driver.

What is the proper way to run spark submit with two jars while one with parameters

To unsubscribe e-mail:

View raw message