hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Ask : Is Hama stores all things in memory ? How to reuse my graph ?
Date Wed, 10 Apr 2013 09:46:34 GMT
On Wed, Apr 10, 2013 at 5:48 PM, 吕雪栋 <lxd.1990@gmail.com> wrote:
> Dear all,
> I am using Hama to implement a search algorithm on a large graph. There are
> some Hama's properties confuse me. Wish you could give me a hand.
>  1. My task is based on a large graph(more than 10T vertices), whether Hama
> stores all the vertices in memory? If so, the memory volume will be a
> limitation for the size of my task, right? I wish you can explain how Hama
> stores its vertices and how to handle the beyond-memory-size graph.

To support memory efficient disk-based vertex storage, Thomas Jungblut
added concept of serial processing, and DiskVerticesInfo
implementation. But, since there is a sorting problem of input
partitioner, we didn't test/release it yet. So you can use only
in-memory style at the moment.

>  2.For my graph is too large, creating vertices and edges will take a long
> time, but the graph is almost stable, my compute() only uses but rewrites or
> changes the value of every vertex. In other words, I want to create my graph
> once and use it multiple times. What should I do?

It looks like your algorithm is related to graph traversal. But sorry,
I'm not sure whether I understand Your question exactly. Can you
please tell me more details?

Basically, Pregel model is "read once, iterate multiple times".

> Thanks for your help.
> Best Regards,
> lv xuedong,
> Tianjin University.

Best Regards, Edward J. Yoon

View raw message