spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: Pyspark: "Error: No main class set in JAR; please specify one with --class"
Date Thu, 01 Oct 2015 17:07:50 GMT
How are you running the actual application?

I find it slightly odd that you're setting PYSPARK_SUBMIT_ARGS
directly; that's supposed to be an internal env variable used by
Spark. You'd normally pass those parameters in the spark-submit (or
pyspark) command line.

On Thu, Oct 1, 2015 at 8:56 AM, YaoPau <jonrgregg@gmail.com> wrote:
> I'm trying to add multiple SerDe jars to my pyspark session.
>
> I got the first one working by changing my PYSPARK_SUBMIT_ARGS to:
>
> "--master yarn-client --executor-cores 5 --num-executors 5 --driver-memory
> 3g --executor-memory 5g --jars /home/me/jars/csv-serde-1.1.2.jar"
>
> But when I tried to add a second, using:
>
> "--master yarn-client --executor-cores 5 --num-executors 5 --driver-memory
> 3g --executor-memory 5g --jars /home/me/jars/csv-serde-1.1.2.jar,
> /home/me/jars/json-serde-1.3-jar-with-dependencies.jar"
>
> I got the error "Error: No main class set in JAR; please specify one with
> --class".
>
> How do I specify the class for just the second JAR?
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Pyspark-Error-No-main-class-set-in-JAR-please-specify-one-with-class-tp24900.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message