giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohammad Kamrul Islam (JIRA)" <>
Subject [jira] [Updated] (GIRAPH-737) Giraph Application Master: move to new and stable YARN API
Date Fri, 01 Nov 2013 01:50:18 GMT


Mohammad Kamrul Islam updated GIRAPH-737:

    Attachment: GIRAPH-737-3.patch

Updated with review comments.
Tested with this two commands:
1. mvn clean verify
2. mvn -Phadoop_yarn -Dhadoop.version=2.2.0 clean verify

> Giraph Application Master: move to new and stable YARN API
> ----------------------------------------------------------
>                 Key: GIRAPH-737
>                 URL:
>             Project: Giraph
>          Issue Type: New Feature
>          Components: mapreduce
>            Reporter: Mohammad Kamrul Islam
>            Assignee: Mohammad Kamrul Islam
>         Attachments: GIRAPH-737-2.patch, GIRAPH-737-3.patch, GIRAPH-737.WIP.patch
> Giraph was the early adopter of Hadoop YARN AM! Eli successfully wrote a Giraph AM based
on Hadoop 2.0.x_alpha. However, in last few months, Yarn significantly *overhauled* its APIs
and associated coding patterns. The new beta version is 2.1.x and I was told by Yarn-dev that
current APIs will not change much.
> In the above circumstances, we need to substantially overhaul Giraph AM as well to accommodate
with the new Yarn API. Moreover, in newer YARN API, supporting kerberos security in AM becomes
easier and more transparent.
> Potential impact:
> The upcoming Girpah AM will not work with earlier alpha Hadoop versions such as 2.0.3.
I'm not sure if anyone is using Giraph AM in production. However, the more prevalent way of
Giraph processing (MR-based) should continue to work.

This message was sent by Atlassian JIRA

View raw message