spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Parhomenko <sparhome...@gmail.com>
Subject Spark does not build with latest version of Hadoop
Date Fri, 27 Sep 2013 15:50:52 GMT
Hi,

We tried building recently released Spark 0.8.0 with latest version of
hadoop-client (2.1.0-beta) and YARN support. According to release
notes<http://spark.incubator.apache.org/releases/spark-release-0-8-0.html>,
0.8.0 should support YARN. The build, however, fails with compilation
error. Command line used was *SPARK_HADOOP_VERSION=2.1.0-beta
SPARK_YARN=true sbt/sbt assembly*, the log file is attached. Latest master
code also fails. Building without YARN or with previous (2.0.6-alpha)
version of hadoop-client works.

Are we doing something wrong or is the support for YARN on 2.1.0-beta not
yet there? We would prefer not to downgrade from Hadoop 2 beta to alpha.

--
Best regards,
Sergey Parhomenko

Mime
View raw message