spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nitinkak001 <nitinkak...@gmail.com>
Subject Generating version agnostic jar path value for --jars clause
Date Fri, 01 May 2015 20:06:27 GMT
I have a list of cloudera jars which I need to provide in --jars clause,
mainly for the HiveContext functionality I am using. However,  many of these
jars have version number as part of their names. This leads to an issue that
the names might change when I do a Cloudera upgrade.

Just a note here, there are many jars which cloudera exposes as a symlink
which is the link to the latest version of that jar(e.g
/opt/cloudera/parcels/CDH/lib/parquet/parquet-hadoop-bundle.jar ->
parquet-hadoop-bundle-1.5.0-cdh5.3.2.jar),in which case its good but there
are many jars which aren't.

Is there a flexible way to avoid this situation?



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Generating-version-agnostic-jar-path-value-for-jars-clause-tp22734.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message