spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Povey <...@maana.io>
Subject Re: Maven repo for Spark pre-built with CDH4?
Date Tue, 18 Mar 2014 23:43:11 GMT
FWIW after searching for the same libraryI had to build spark it to get it to work with HDFS
on a cloudera install

I downloaded the CDH version from the spark site and still had to build it to get it to work,
this is the command I used


SPARK_HADOOP_VERSION=2.0.0-cdh4.6.0 sbt/sbt assembly
SPARK_HADOOP_VERSION=2.0.0-cdh4.6.0 sbt/sbt publish-local

I’ve run both Spark-Shell remotely and applications linked against the built library



Rob Povey
rob@maana.io<mailto:rob@maana.io>



On Mar 18, 2014, at 1:55 PM, Punya Biswal <pbiswal@palantir.com<mailto:pbiswal@palantir.com>>
wrote:

Hi all,

The Maven central repo contains an artifact for spark 0.9.0 built with unmodified Hadoop,
and the Cloudera repo contains an artifact for spark 0.9.0 built with CDH 5 beta. Is there
a repo that contains spark-core built against a non-beta version of CDH (such as 4.4.0)?

Punya



Mime
View raw message