spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <matei.zaha...@gmail.com>
Subject Re: spark-0.8.0 and hadoop-2.1.0-beta
Date Sun, 29 Sep 2013 15:59:43 GMT
Hi Terence,

YARN's API changed in an incompatible way in Hadoop 2.1.0, so I'd suggest sticking with 2.0.x
for now. We may create a different branch for this version. Unfortunately due to the API change
it may not be possible to support this version while also supporting other widely-used versions
like 0.23.x.

Matei

On Sep 29, 2013, at 11:00 AM, Terance Dias <terance.dias@gmail.com> wrote:

> 
> Hi, I'm trying to build spark-0.8.0 with hadoop-2.1.0-beta.
> I have changed the following properties in SparkBuild.scala file.
> 
> val DEFAULT_HADOOP_VERSION = "2.1.0-beta"
> val DEFAULT_YARN = true
> 
> when i do sbt clean compile, I get an error saying
> 
> [error] /usr/local/spark-0.8.0-incubating/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala:42:
not found: type AMRMProtocol
> [error]   private var resourceManager: AMRMProtocol = null
> 
> Thanks,
> Terance.
> 


Mime
View raw message