spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel S Horwitz <horw...@avast.com>
Subject Re: Spark does not build with latest version of Hadoop
Date Fri, 27 Sep 2013 16:16:23 GMT
How do I unsubscribe?

Thanks,

Joel S Horwitz | Director of Analytics
www.avast.com | horwitz@avast.com
US: +1(415)590-0997 | CZ: +420 274 005 876


On Sep 27, 2013, at 8:50 AM, Sergey Parhomenko <sparhomenko@gmail.com> wrote:

> Hi,
> 
> We tried building recently released Spark 0.8.0 with latest version of hadoop-client
(2.1.0-beta) and YARN support. According to release notes, 0.8.0 should support YARN. The
build, however, fails with compilation error. Command line used was SPARK_HADOOP_VERSION=2.1.0-beta
SPARK_YARN=true sbt/sbt assembly, the log file is attached. Latest master code also fails.
Building without YARN or with previous (2.0.6-alpha) version of hadoop-client works.
> 
> Are we doing something wrong or is the support for YARN on 2.1.0-beta not yet there?
We would prefer not to downgrade from Hadoop 2 beta to alpha.
> 
> --
> Best regards,
> Sergey Parhomenko
> <2.1.0-beta-YARN.log>


Mime
View raw message