spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Building Spark with Custom Hadoop Version
Date Thu, 04 Feb 2016 23:11:53 GMT
Assuming your change is based on hadoop-2 branch, you can use 'mvn install'
command which would put artifacts under 2.8.0-SNAPSHOT subdir in your local
maven repo.

Here is an example:
~/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.8.0-SNAPSHOT

Then you can use the following command to build Spark:

-Pyarn -Phadoop-2.4 -Dhadoop.version=2.8.0-SNAPSHOT

FYI

On Thu, Feb 4, 2016 at 3:03 PM, Charles Wright <charliewright@live.ca>
wrote:

> Hello,
>
> I have made some modifications to the YARN source code that I want to test
> with Spark, how do I do this? I know that I need to include my custom
> hadoop jar as a dependency but I don't know how to do this as I am not very
> familiar with maven.
>
> Any help is appreciated.
>
> Thanks,
> Charles.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
View raw message