hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Notes for working on mapreduce trunk after the MR-279 merge.
Date Thu, 18 Aug 2011 15:46:39 GMT
It looks like git has not seen the changes yet, even though the last change was over 90 mins
ago.  Is there any way to kick git to pull in the changes sooner so I can rebase?


Bobby Evans

On 8/18/11 7:49 AM, "Vinod Kumar Vavilapalli" <vinodkv@hortonworks.com> wrote:

MR-279 branch is merged into mapreduce trunk and this changes things a
bit for developing on mapreduce.

You can get all the help that is needed from the INSTALL file at
Reproducing some of those contents here for the short-term lookup.

Checking out source code
svn checkout http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce

Directory structure

  - hadoop-mapreduce ( was mapreduce before)

trunk/hadoop-mapreduce - Classic code. JT/TT reside here
 - build.xml
 - src

trunk/hadoop-mapreduce/ - New code related to yarn reside here.
 - assembly
 - pom.xml
 - hadoop-mr-client
 - hadoop-yarn - Yarn APIs, libraries, and server code
   -- hadoop-yarn-api
   -- hadoop-yarn-common
   -- hadoop-yarn-server - Server code, ResourceManager, NodeManager,
server libraries and tests.
      --- hadoop-yarn-server-common
      --- hadoop-yarn-server-nodemanager
      --- hadoop-yarn-server-resourcemanager
      --- hadoop-yarn-server-tests
 - hadoop-mr-client - MapReduce server and client code
   -- hadoop-mapreduce-client-app
   -- hadoop-mapreduce-client-core
   -- hadoop-mapreduce-client-jobclient
   -- hadoop-mapreduce-client-common
   -- hadoop-mapreduce-client-hs
   -- hadoop-mapreduce-client-shuffle

Building yarn code and install into the local maven cache.
 - mvn clean install
 - In case you want to skip the tests run: mvn clean install -DskipTests

Building classic code once yarn code is built.
 - ant veryclean jar jar-test  -Dresolvers=internal

 1) For hacking on the new yarn+MR code in eclipse, you should run
"mvn eclipse:eclipse" and then import the checked out source root as a
maven project.
 2) For developing on classic JT/TT code, running "ant eclipse" and
importing as java project should continue to work.

Hope that helps. If you run into issues, please send an email or
create a JIRA issue.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message