hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Pagerank example doesn't work
Date Thu, 30 Aug 2012 13:43:46 GMT
Yes, I'll try it.

On Thu, Aug 30, 2012 at 10:38 PM, Thomas Jungblut
<thomas.jungblut@gmail.com> wrote:
> Still has some overhead, the wiki dataset I benched were just 1gb.
> Wonder where our hotspots are, did you get your yourkit license to profile?
>
> 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
>
>> My settings and envs were like this:
>>
>>  * child.java.opts = -Xmx1024m
>>  * tasks.num per node = 5
>>  * total tasks = 85
>>  * network env = infiniband 40G
>>  * physical nodes = 17
>>  * hadoop-cdh-0.20
>>  * zookeeper-cdh
>>
>> Finished in 127 secs.
>>
>> On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut
>> <thomas.jungblut@gmail.com> wrote:
>> > Yeah, lots of ram ;))
>> >
>> > 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
>> >
>> >> > It runs locally in under 120s on my computer
>> >>
>> >> Haha, single machine?
>> >>
>> >> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
>> >> <thomas.jungblut@gmail.com> wrote:
>> >> > It runs locally in under 120s on my computer so I assume that this
is
>> >> > correct.
>> >> >
>> >> > 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
>> >> >
>> >> >> Just two minutes were needed to process wikipedia dataset. Is this
>> >> normal?
>> >> >>
>> >> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
>> >> >> <thomas.jungblut@gmail.com> wrote:
>> >> >> > http://snap.stanford.edu/data/
>> >> >> >
>> >> >> > 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
>> >> >> >
>> >> >> >> Just wondering, do you know more large and downloadable
data set?
>> >> >> >>
>> >> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
>> >> edwardyoon@apache.org>
>> >> >> >> wrote:
>> >> >> >> > Oh... thanks. now I can guess what it is.
>> >> >> >> >
>> >> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
>> >> >> >> > <thomas.jungblut@gmail.com> wrote:
>> >> >> >> >> Yes, the example graph is not complete thus resulting
in NPEs.
>> >> >> >> >>
>> >> >> >> >> 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
>> >> >> >> >>
>> >> >> >> >>> Nope, should I set to true?
>> >> >> >> >>>
>> >> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
>> >> >> >> >>> <thomas.jungblut@gmail.com> wrote:
>> >> >> >> >>> > Set "hama.graph.repair" to true in conf?
>> >> >> >> >>> >
>> >> >> >> >>> > 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
>> >> >> >> >>> >
>> >> >> >> >>> >> I installed Hama 0.5 on 4 racks
(4 x 18 nodes) w/ 3 tasks
>> per
>> >> >> each
>> >> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m),
tried to run
>> PageRank
>> >> on
>> >> >> >> >>> >> Wikipedia dataset introduced at
>> >> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
>> >> >> >> >>> >>
>> >> >> >> >>> >> Doesn't work, the cause of job fail
error is different
>> every
>> >> >> time.
>> >> >> >> >>> >> NullPointerException at loadVertices(),
ERROR bsp.BSPTask:
>> >> >> Started
>> >> >> >> >>> >> pinging to groom, ..., etc.
>> >> >> >> >>> >>
>> >> >> >> >>> >> Has anyone tried this?
>> >> >> >> >>> >>
>> >> >> >> >>> >> --
>> >> >> >> >>> >> Best Regards, Edward J. Yoon
>> >> >> >> >>> >> @eddieyoon
>> >> >> >> >>> >>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> --
>> >> >> >> >>> Best Regards, Edward J. Yoon
>> >> >> >> >>> @eddieyoon
>> >> >> >> >>>
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Best Regards, Edward J. Yoon
>> >> >> >> > @eddieyoon
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Best Regards, Edward J. Yoon
>> >> >> >> @eddieyoon
>> >> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Best Regards, Edward J. Yoon
>> >> >> @eddieyoon
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>> >> @eddieyoon
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message