giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohammad Islam" <misla...@yahoo.com>
Subject Re: Review Request 14575: Giraph Application Master: move to new and stable YARN API
Date Fri, 01 Nov 2013 01:48:52 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14575/
-----------------------------------------------------------

(Updated Nov. 1, 2013, 1:48 a.m.)


Review request for giraph.


Changes
-------

Updated with review comments.
Tested with this two commands:
1. mvn clean verify
2. mvn -Phadoop_yarn -Dhadoop.version=2.2.0 clean verify


Bugs: GIRAPH-737
    https://issues.apache.org/jira/browse/GIRAPH-737


Repository: giraph-git


Description
-------

WIP patch. Please give your early comments.

Key points:

* Giraph AM using new API and asynchronous/handler model.
* Adding Kerberos support.

Copied from the JIRA:

Giraph was the early adopter of Hadoop YARN AM! Eli successfully wrote a Giraph AM based on
Hadoop 2.0.x_alpha. However, in last few months, Yarn significantly overhauled its APIs and
associated coding patterns. The new beta version is 2.1.x and I was told by Yarn-dev that
current APIs will not change much.
In the above circumstances, we need to substantially overhaul Giraph AM as well to accommodate
with the new Yarn API. Moreover, in newer YARN API, supporting kerberos security in AM becomes
easier and more transparent.
Potential impact:
The upcoming Girpah AM will not work with earlier alpha Hadoop versions such as 2.0.3. I'm
not sure if anyone is using Giraph AM in production. However, the more prevalent way of Giraph
processing (MR-based) should continue to work.


Diffs (updated)
-----

  README fd9ec20 
  findbugs-exclude.xml 21aa4ef 
  giraph-core/src/main/java/org/apache/giraph/comm/netty/SaslNettyServer.java 00a802f 
  giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/SaslServerHandler.java 922f373

  giraph-core/src/main/java/org/apache/giraph/yarn/GiraphApplicationMaster.java c2b88a0 
  giraph-core/src/main/java/org/apache/giraph/yarn/GiraphYarnClient.java 341db0e 
  giraph-core/src/main/java/org/apache/giraph/yarn/YarnUtils.java aa042e8 
  pom.xml f2981ff 

Diff: https://reviews.apache.org/r/14575/diff/


Testing
-------


Thanks,

Mohammad Islam


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message