mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Mannix <jake.man...@gmail.com>
Subject Re: LDA/CVB Performance
Date Thu, 13 Jun 2013 21:44:14 GMT
Yep, agreed, try whatever works, given the env!  If in the course of
debugging,
someone can tell where I did something dumb with the mutlithreading,
I'd be happy to dig in and fix it, but multhreaded *performance* is one of
those
things that's really hard to fix with a unit test, so I've never really
gotten to the
bottom of why num_train_threads doesn't seem to speed it up as much as it
should.


On Thu, Jun 13, 2013 at 2:37 PM, Sebastian Schelter <ssc@apache.org> wrote:

> > I'm not too much of a fan of stealing control of the whole box - my local
> > hadoop admin would really not like me. :)
>
> Completely agree. Our implementations should not do this, thats why ALS
> runs per default with a single thread per mapper.
>
> Just wanted to point out that there are some tricks one can play to
> greatly enhance performance, given the environment (e.g. spawned cluster
> on EC2, research setting) supports them.
>
> -sebastian
>
>
>


-- 

  -jake

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message