mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: Mahout running on Hadoop
Date Mon, 05 Jul 2010 09:13:36 GMT
In general, the Hadoop-based implementations are completely different
creatures. Code from the regular online versions doesn't port and the
computation needs to be structured quite differently. They're almost
different libraries.

There's one hybrid, and that is the pseudo-distributed recommender
bits in org.apache.mahout.cf.taste.hadoop.pseudo. This is a way to run
many non-distributed normal Recommenders on Hadoop. The computation
isn't actually distributed; it's just that many instances are run. The
issue there is eventually your data outgrows what can be loaded on one
machine, but, could be useful.

Otherwise, no there's no way to just move a given implementation to
Hadoop in a fully distributed way. Some algorithms just won't be
distributable.

On Sun, Jul 4, 2010 at 9:00 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
> The recommendation capabilities are the best integrated and most
> interchangeable parts of Mahout.
>
> You should be able to start with entirely on-line recommendations and switch
> to off-line methods fairly transparently as you scale.  In addition, you
> should be able to use off-line precomputation with hadoop and still use
> non-hadoop based methods for experiments.
>
> Sean should probably comment on the details, but I am pretty sure that the
> statement above is a good summary.
>
> 2010/7/4 Matthias Böhmer <matthias@m-boehmer.de>
>
>> Yes, right! I have an non-Hadoop implementation using the API and I am
>> wondering which steps I have to take to move to a Hadoop-based
>> implementation. It seems like I have to change my application code,
>> right? Or is there a way to keep my application code as it is, e.g.
>> for running tests without Hadoop.
>>
>> 2010/7/2 Ted Dunning <ted.dunning@gmail.com>:
>> > By this, do you mean migrate from using the Mahout recommendation
>> framework
>> > without hadoop to using the Mahout recommendation framework with Hadoop?
>> >
>> > On Fri, Jul 2, 2010 at 8:26 AM, <matboehmer@googlemail.com> wrote:
>> >
>> >> However, I am currently looking for an easy way of how to migrate to
>> >> Hadoop.
>> >
>>
>>
>>
>> --
>> --
>>
>

Mime
View raw message