giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Presta <>
Subject Re: GiraphFileInputFormat questions
Date Fri, 08 Feb 2013 23:06:16 GMT
Hi Eli,

Yes, GiraphFileInputFormat deals with input splitting in all cases. Note
that most of the logic is the same as in current Hadoop, and we extend
Hadoop's FileInputFormat.
I wish there was a way to avoid any code duplication, but this is messing
with implementation-specific code that is mostly private.


On 2/8/13 2:58 PM, "Eli Reisman" <> wrote:

>Hey (maybe @Alessandro, don't know...) I have been looking at the
>GiraphFileInputFormat. Am I crazy, or with the advent of edge or vertex
>based input files, do we now always generate our own input splits, from
>scratch, without hadoop being involved? And if so, is this defaulted to
>"on" no matter what, or only when we have dual edge-vertex input
>information to process? If so, its one less thing I will have to implement
>for the YARN implementation.
>Thanks, looking forward to hearing back,

View raw message