spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Botorek <Jan.Boto...@infor.com>
Subject Add jar files on classpath when submitting tasks to Spark
Date Tue, 01 Nov 2016 12:11:45 GMT
Hello,
I have a problem trying to add jar files to be available on classpath when submitting task
to Spark.

In my spark-defaults.conf file I have configuration:
spark.driver.extraClassPath = path/to/folder/with/jars
all jars in the folder are available in SPARK-SHELL

The problem is that jars are not on the classpath for SPARK-MASTER; more precisely - when
I submit any job that utilizes any jar from external folder, the java.lang.ClassNotFoundException
is thrown.
Moving all external jars into the jars folder solves the situation, but we need to keep external
files separatedly.

Thank you for any help
Best regards,
Jan

Mime
View raw message