giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Armando Miraglia (JIRA)" <>
Subject [jira] [Commented] (GIRAPH-732) EdgeOutputFormat
Date Thu, 08 Aug 2013 11:54:48 GMT


Armando Miraglia commented on GIRAPH-732:

Hi Nitay,

very interesting! I was actually thinking the same a couple of days ago while working on the
issue GIRAPH-723.

Now, since there is the plan to do this I would like to ask you whether it is possible to
add facilities to make these two conditions satisfied:
1. Guarantee that VertexOutputFormat is executed _before_ EdgeOutputFormat in the case both
are provided.
2. Make the afore mentioned condition valid also globally for all the workers. This means
that all workers that have already wirtting using the VertexOutputFormat will wait until all
the other workers are done before sending the edges to the output endpoint.

These two conditions would make the implementation of output formats easier and in this case
it would make the implementation of GIRAPH-723 very clean.
The reason for these conditions is that when the output in which the graph is going to be
saved requires the edges to be created among _already existing_ vertices (for example in case
of a graphdb), there would be a way to guarantee that vertices are in fact already present
in the database. This would result in an efficient output format implementation.
Differently, a process of Map/Reduce which requires the resulting graph to be collected first
needs to be implemented or on the endpoint side there must be some logic to deal with the
temporary inconsistent graph.

> EdgeOutputFormat
> ----------------
>                 Key: GIRAPH-732
>                 URL:
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Nitay Joffe
> Outputting edges is as easy as looping through edges in a vertex and writing them out.
However right now the user still needs to write a VertexOutputFormat and do it themselves.
> We should make this easier for them by adding an EdgeOutputFormat API and allowing the
user to specify a VertexOutputFormat and/or an EdgeOutputFormat.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message