hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: loading vertices into RAM
Date Sun, 19 Jan 2014 06:56:57 GMT
yes, the correct way of setting OffHeapVI is: conf.setClass("
hama.graph.vertices.info",org.apache.hama.graph.
OffHeapVerticesInfo.class,org.apache.hama.graph.VerticesInfo.class);

Apart from that, what Hama version are you running on?
Looking at the code in trunk it shouldn't be possible to have a NPE on the
currentVertex if the iterator is consumed correctly, instead if one doesn't
call hasNext before next and / or calls next even if hasNext returns false
then it's possible to have that NPE.
Also what algorithm / example are you running? Any useful information (like
environment, execution mode, logs, version, etc.) would be useful to help
you.

Tommaso



2014/1/19 步青云 <mailliuping@qq.com>

> I got the same problem about loading vertices into RAM.And I try to use
> off OffHeapVerticesInfo.
> You may use the method setClass like this:
> conf.setClass("hama.graph.vertices.info
> ",org.apache.hama.graph.OffHeapVerticesInfo.class,org.apache.hama.graph.VerticesInfo.class);
> However,I got the Nullexception using OffHeapVerticesInfo.The errors are
> as follows:
>
> 14/01/18 20:54:23 ERROR bsp.LocalBSPRunner: Exception during BSP execution!
> java.lang.NullPointerException
>     at
> org.apache.hama.graph.OffHeapVerticesInfo$1.next(OffHeapVerticesInfo.java:139)
>     at
> org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:251)
>     at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:145)
>     at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:256)
>     at
> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:286)
>
> Anyone could help me to solve this problem?Thanks a lot.
>
>
>
>
> ------------------ Original ------------------
> From:  "Ammar Sahib"<ammar.sahib@yahoo.com>;
> Date:  Jan 17, 2014
> To:  "user@hama.apache.org"<user@hama.apache.org>;
>
> Subject:  Re: loading vertices into RAM
>
>
>
> I think we are getting close now, However now I have runtime exception:
>
> Exception in thread "main" java.lang.RuntimeException: interface
> org.apache.hama.graph.VerticesInfo not
> org.apache.hama.graph.ListVerticesInfo
>     at
> org.apache.hadoop.conf.Configuration.setClass(Configuration.java:858)
>
>
>
>
>
> On Friday, January 17, 2014 2:30 PM, Tommaso Teofili <
> tommaso.teofili@gmail.com> wrote:
>
> ah yes, sorry, you also have to specify the interface, I don't have the
> code in front of me but it should be :
>
> conf.setClass("hama.graph.vertices.info",
> org.apache.hama.graph.VerticesInfo.class, org.apache.
> hama.graph.ListVerticesInfo.class);
>
> Tommaso
>
>
>
> 2014/1/17 Ammar Sahib <ammar.sahib@yahoo.com>
>
> > Hi
> >
> > Thanks for your reply. I used now:
> >
> > conf.setClass("hama.graph.vertices.info
> > ",org.apache.hama.graph.ListVerticesInfo.class);
> >
> > Now I get this error:
> > The method setClass(String, Class<?>, Class<?>) in the type Configuration
> > is not applicable for the arguments (String, Class<ListVerticesInfo>)
> >
> > I am using HAMA 0.6.3
> >
> >
> >
> >
> >
> > On Friday, January 17, 2014 12:59 PM, Tommaso Teofili <
> > tommaso.teofili@gmail.com> wrote:
> >
> > you're passing the fully qualified name of the Class as a String to a
> > method setClass(String, Class) while you should pass the Class itself,
> > e.g.:
> > HamaConfiguration conf = new HamaConfiguration();
> > conf.setClass("hama.graph.vertices.info",org.apache.
> > hama.graph.ListVerticesInfo.class);
> >
> > Hope this helps,
> > Tommaso
> >
> >
> >
> >
> > 2014/1/17 Ammar Sahib <ammar.sahib@yahoo.com>
> >
> > > Hi
> > >
> > > I am trying to evaluate the different implementation below:
> > >
> > >
> > > - ListVerticesinfo: loads vertices into array list.
> > > - MapVerticesinfo: loads vertices into tree map.
> > > - DiskVerticesInfo: loads vertices into a local file.
> > >
> > > When using the conf.setClass method I got an error. Below is sample of
> my
> > > code:
> > > HamaConfiguration conf = new HamaConfiguration();
> > > conf.setClass("hama.graph.vertices.info
> > > ","org.apache.hama.graph.ListVerticesInfo");
> > >
> > > The error I am getting is:
> > > The method setClass(String, Class<?>, Class<?>) in the type
> Configuration
> > > is not applicable for the arguments (String, String).
> > >
> > > However I found that I can use conf.set method.
> > >
> > >
> > > Can someone tell me what is I am doing wrong?
> > >
> > >
> > >
> > >
> > >
> > > On Wednesday, January 15, 2014 8:01 AM, Tommaso Teofili <
> > > tommaso.teofili@gmail.com> wrote:
> > >
> > > and OffHeapVerticesInfo for loading vertices off heap, which is
> available
> > > with 0.6.3 as well if I recall correctly.
> > > Tommaso
> > >
> > >
> > >
> > > 2014/1/15 Edward J. Yoon <edwardyoon@apache.org>
> > >
> > > > There are few implementations.
> > > >
> > > >  - ListVerticesinfo: loads vertices into array list.
> > > >  - MapVerticesinfo: loads vertices into tree map.
> > > >  - DiskVerticesInfo: loads vertices into a local file.
> > > >
> > > > You can choose one of them by setting the "hama.graph.vertices.info"
> > > > in job configuration.
> > > >
> > > >   > conf.setClass("hama.graph.vertices.info",
> > > > "org.apache.hama.graph.ListVerticesInfo".
> > > >
> > > > With the latest 0.6.3 version, you can use only ListVerticesInfo.
> > > > Please use the TRUNK.
> > > >
> > > >
> > > > On Tue, Jan 14, 2014 at 11:18 PM, Ammar Sahib <ammar.sahib@yahoo.com
> >
> > > > wrote:
> > > > > Hi
> > > > >
> > > > > According to the BSP model, the data is processed in the RAM and
> that
> > > is
> > > > the reason why Pregel model is faster than the MapReduce (MapReduce
> > > > writedown to disk). Can someone explains to me how to be sure that
> all
> > > the
> > > > graph vertices are actually been loaded in RAM?
> > > > >
> > > > >
> > > > > How would HAMA behave if the vertices values are so big such that
> the
> > > > available RAM memory is not enough to contains all of the vertices?
> > > > >
> > > > > Regards
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards, Edward J. Yoon
> > > > @eddieyoon
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message