spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Parhomenko <>
Subject Spark does not build with latest version of Hadoop
Date Fri, 27 Sep 2013 15:50:52 GMT

We tried building recently released Spark 0.8.0 with latest version of
hadoop-client (2.1.0-beta) and YARN support. According to release
0.8.0 should support YARN. The build, however, fails with compilation
error. Command line used was *SPARK_HADOOP_VERSION=2.1.0-beta
SPARK_YARN=true sbt/sbt assembly*, the log file is attached. Latest master
code also fails. Building without YARN or with previous (2.0.6-alpha)
version of hadoop-client works.

Are we doing something wrong or is the support for YARN on 2.1.0-beta not
yet there? We would prefer not to downgrade from Hadoop 2 beta to alpha.

Best regards,
Sergey Parhomenko

View raw message