giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohammad Islam" <>
Subject Re: Review Request 14575: Giraph Application Master: move to new and stable YARN API
Date Fri, 01 Nov 2013 01:48:52 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Nov. 1, 2013, 1:48 a.m.)

Review request for giraph.


Updated with review comments.
Tested with this two commands:
1. mvn clean verify
2. mvn -Phadoop_yarn -Dhadoop.version=2.2.0 clean verify

Bugs: GIRAPH-737

Repository: giraph-git


WIP patch. Please give your early comments.

Key points:

* Giraph AM using new API and asynchronous/handler model.
* Adding Kerberos support.

Copied from the JIRA:

Giraph was the early adopter of Hadoop YARN AM! Eli successfully wrote a Giraph AM based on
Hadoop 2.0.x_alpha. However, in last few months, Yarn significantly overhauled its APIs and
associated coding patterns. The new beta version is 2.1.x and I was told by Yarn-dev that
current APIs will not change much.
In the above circumstances, we need to substantially overhaul Giraph AM as well to accommodate
with the new Yarn API. Moreover, in newer YARN API, supporting kerberos security in AM becomes
easier and more transparent.
Potential impact:
The upcoming Girpah AM will not work with earlier alpha Hadoop versions such as 2.0.3. I'm
not sure if anyone is using Giraph AM in production. However, the more prevalent way of Giraph
processing (MR-based) should continue to work.

Diffs (updated)

  README fd9ec20 
  findbugs-exclude.xml 21aa4ef 
  giraph-core/src/main/java/org/apache/giraph/comm/netty/ 00a802f 
  giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/ 922f373

  giraph-core/src/main/java/org/apache/giraph/yarn/ c2b88a0 
  giraph-core/src/main/java/org/apache/giraph/yarn/ 341db0e 
  giraph-core/src/main/java/org/apache/giraph/yarn/ aa042e8 
  pom.xml f2981ff 




Mohammad Islam

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message