giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-503) Refactor platform-independent CLI argument parsing in GiraphRunner into a separate class
Date Fri, 08 Feb 2013 00:43:12 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13574110#comment-13574110
] 

Eli Reisman commented on GIRAPH-503:
------------------------------------

One refactor I will put up a JIRA for when this is committed, is to move the "configuration
validation" code out of GiraphJob (perhaps a dual refactor of GiraphTypeValidator too) so
that our conf checking, like our arg parsing, are modules of their own and can be called easily
from any Runner etc. without duplication.

We could make a generic Runner for Hive, Yarn, and Hadoop giraph versions from the one this
patch leaves us with by making custom GiraphJob impls instead, and perhaps a no-op runner
method that can be overridden to add the extra config options for parsing, as discussed in
the earlier thread post, using the new method I just added to CliParserUtils. And a no-op
for platform dependent setup, if needed? Then HiveGiraphRunner could just inherit from GiraphRunner,
and implement its setup code in these method overrides?

Anyway, there's time to figure it out; lots of ways to fix these up once this is in...

                
> Refactor platform-independent CLI argument parsing in GiraphRunner into a separate class
> ----------------------------------------------------------------------------------------
>
>                 Key: GIRAPH-503
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-503
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Eli Reisman
>            Assignee: Eli Reisman
>            Priority: Minor
>         Attachments: GIRAPH-503-1.patch, GIRAPH-503-2.patch, GIRAPH-503-3.patch, GIRAPH-503-4.patch
>
>
> In order to run on non Hadoop MR platforms, we will need to populate the GiraphConfiguration
for our job in a platform-independent way so that all config options are available to whatever
driver class initiates the Giraph job (not just GiraphRunner/GiraphJob.) This also serves
to clean up GiraphRunner in general.
> Passes 'mvn clean install'
> Review Board URL: https://reviews.apache.org/r/9350/

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message