spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nitinkak001 <>
Subject Generating version agnostic jar path value for --jars clause
Date Fri, 01 May 2015 20:06:27 GMT
I have a list of cloudera jars which I need to provide in --jars clause,
mainly for the HiveContext functionality I am using. However,  many of these
jars have version number as part of their names. This leads to an issue that
the names might change when I do a Cloudera upgrade.

Just a note here, there are many jars which cloudera exposes as a symlink
which is the link to the latest version of that jar(e.g
/opt/cloudera/parcels/CDH/lib/parquet/parquet-hadoop-bundle.jar ->
parquet-hadoop-bundle-1.5.0-cdh5.3.2.jar),in which case its good but there
are many jars which aren't.

Is there a flexible way to avoid this situation?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message