spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: netty on classpath when using spark-submit
Date Thu, 04 Dec 2014 06:13:12 GMT
Markus,

On Tue, Nov 11, 2014 at 10:40 AM, M. Dale <medale94@yahoo.com> wrote:
>
>   I never tried to use this property. I was hoping someone else would jump
> in. When I saw your original question I remembered that Hadoop has
> something similar. So I searched and found the link below. A quick JIRA
> search seems to indicate that there is another property:
>
> https://issues.apache.org/jira/browse/SPARK-2996
>
> Maybe that property will work with yarn: spark.yarn.user.classpath.first
>

Thank you very much! That property does in fact load the classes from my
jar file first when running on YARN, great!

However, in local[N] mode, neither that one nor
the spark.files.userClassPathFirst one works. So when using spark-submit
with "--master local[3]" instead of "--master yarn-cluster", the value
for spark.files.userClassPathFirst is displayed correctly, but the classes
are still loaded from the wrong jar...

Tobias

Mime
View raw message