spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <>
Subject Re: Building Spark with Custom Hadoop Version
Date Thu, 04 Feb 2016 23:11:53 GMT
Assuming your change is based on hadoop-2 branch, you can use 'mvn install'
command which would put artifacts under 2.8.0-SNAPSHOT subdir in your local
maven repo.

Here is an example:

Then you can use the following command to build Spark:

-Pyarn -Phadoop-2.4 -Dhadoop.version=2.8.0-SNAPSHOT


On Thu, Feb 4, 2016 at 3:03 PM, Charles Wright <>

> Hello,
> I have made some modifications to the YARN source code that I want to test
> with Spark, how do I do this? I know that I need to include my custom
> hadoop jar as a dependency but I don't know how to do this as I am not very
> familiar with maven.
> Any help is appreciated.
> Thanks,
> Charles.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message