spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars Francke <>
Subject Re: YARN Maven build questions
Date Fri, 28 Feb 2014 17:41:00 GMT

so currently it doesn't work because of

IntelliJ reports a lot of warnings with default settings and I haven't
found a way to tell IntellJ to use different Hadoop versions yet.
mvn clean compile -Pyarn fails as well (compilation errror

Your command works indeed. Default yarn version is 0.23.7 which
doesn't seem to work with the default 2.2.0 Hadoop version (anymore?)

I was basically trying to follow the documentation:

mvn clean compile -Pyarn-alpha -Dhadoop.version=2.0.0-cdh4.5.0
-Dyarn.version=2.0.0-cdh4.5.0 fails as well as does mvn clean compile

Thanks for showing me a configuration that works. Unfortunately the
default ones and at least one of the documented ones fail.


On Fri, Feb 28, 2014 at 3:05 PM, Tom Graves <> wrote:
> what build command are you using?    What do you mean when you say YARN branch?
> The yarn builds have been working fine for me with maven.   Build command I use against
hadoop 2.2 or higher: mvn -Dyarn.version=2.2.0 -Dhadoop.version=2.2.0 -Pyarn clean package
> Tom
> On Friday, February 28, 2014 6:14 AM, Lars Francke <> wrote:
> Hey,
> I'm trying to dig into Spark's code but am running into a couple of problems.
> 1) The yarn-common directory is not included in the Maven build
> causing things to fail because the dependency is missing. If I see the
> history correct it used to be a Maven module but is not anymore.
> 2) When I try to include the yarn-common directory in the build things
> start going bad. Compilation failures all over the place and I think
> there are some dependency issues in there as well.
> This leads me to believe that either the Maven build system isn't
> maintained for YARN or the whole YARN branch isn't. What's the status
> here?
> Without YARN things build fine for me using Maven.
> Thanks for your help.
> Cheers,
> Lars

View raw message