spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <>
Date Thu, 16 Oct 2014 00:53:00 GMT
Hi Greg,

I'm not sure exactly what it is that you're trying to achieve, but I'm
pretty sure those variables are not supposed to be set by users. You
should take a look at the documentation for
"spark.driver.extraClassPath" and "spark.driver.extraLibraryPath", and
the equivalent options for executors. (The driver ones also have
direct equivalents in the spark-submit command line, check its help

Since you're running on Yarn, you might also want to take a look at (and SPARK-1720).

On Tue, Oct 14, 2014 at 11:57 AM, Greg Hill <> wrote:
> It seems to me that SPARK_SUBMIT_CLASSPATH does not follow the same ability
> as other tools to put wildcards in the paths you add.  For some reason it
> doesn't pick up the classpath information from yarn-site.xml either, it
> seems, when running on YARN.  I'm having to manually add every single
> dependency JAR.  There must be a better way, so what am I missing?
> Greg


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message