spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Povey <>
Subject Re: Maven repo for Spark pre-built with CDH4?
Date Tue, 18 Mar 2014 23:43:11 GMT
FWIW after searching for the same libraryI had to build spark it to get it to work with HDFS
on a cloudera install

I downloaded the CDH version from the spark site and still had to build it to get it to work,
this is the command I used

SPARK_HADOOP_VERSION=2.0.0-cdh4.6.0 sbt/sbt assembly
SPARK_HADOOP_VERSION=2.0.0-cdh4.6.0 sbt/sbt publish-local

I’ve run both Spark-Shell remotely and applications linked against the built library

Rob Povey<>

On Mar 18, 2014, at 1:55 PM, Punya Biswal <<>>

Hi all,

The Maven central repo contains an artifact for spark 0.9.0 built with unmodified Hadoop,
and the Cloudera repo contains an artifact for spark 0.9.0 built with CDH 5 beta. Is there
a repo that contains spark-core built against a non-beta version of CDH (such as 4.4.0)?


View raw message