We tried building recently released Spark 0.8.0 with latest version of hadoop-client (2.1.0-beta) and YARN support. According to release notes
, 0.8.0 should support YARN. The build, however, fails with compilation error. Command line used was SPARK_HADOOP_VERSION=2.1.0-beta SPARK_YARN=true sbt/sbt assembly
, the log file is attached. Latest master code also fails. Building without YARN or with previous (2.0.6-alpha) version of hadoop-client works.
Are we doing something wrong or is the support for YARN on 2.1.0-beta not yet there? We would prefer not to downgrade from Hadoop 2 beta to alpha.