mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Schilling <ch...@cellixis.com>
Subject Re: Can all the algorithms in Mahout be run locally without a Hadoop cluster.
Date Sat, 25 Jun 2011 02:39:49 GMT
There are nice tutorials to setup Mahout on amazons elastic map-reduce. It's pretty cheap.


I don't have the links in front of me...

On Jun 24, 2011, at 7:21 PM, "XiaoboGu" <guxiaobo1982@gmail.com> wrote:

> I have found this, will this configuration start the corresponding task trackers too?
> 
> http://hadoop-karma.blogspot.com/2010/05/hadoop-cookbook-4-how-to-run-multiple.html
> 
> 
>> -----Original Message-----
>> From: Ted Dunning [mailto:ted.dunning@gmail.com]
>> Sent: Saturday, June 25, 2011 10:12 AM
>> To: dev@mahout.apache.org
>> Cc: user@mahout.apache.org
>> Subject: Re: Can all the algorithms in Mahout be run locally without a Hadoop cluster.
>> 
>> I have done this with VM's but I would not generally recommend it.  Without
>> VM's you will have a pretty ugly configuration issue because Hadoop usually
>> assumes it owns the machine.
>> 
>> Besides, this is a seriously square peg into a round hole kind of problem
>> here.  Hadoop (map-reduce) was designed so that you could use several little
>> machines instead of one big one.  It just isn't going to work well on a
>> single computer.
>> 
>> On Fri, Jun 24, 2011 at 6:49 PM, XiaoboGu <guxiaobo1982@gmail.com> wrote:
>> 
>>> Do you have any experience  in running multiple data nodes and task
>>> trackers on a single SMP server.
>>> 
>>>> -----Original Message-----
>>>> From: Ted Dunning [mailto:ted.dunning@gmail.com]
>>>> Sent: Saturday, June 25, 2011 9:26 AM
>>>> To: user@mahout.apache.org
>>>> Cc: dev@mahout.apache.org
>>>> Subject: Re: Can all the algorithms in Mahout be run locally without a
>>> Hadoop cluster.
>>>> 
>>>> Pretty big.  SHould scream for local classifier learning.
>>>> 
>>>> Local Hadoop should run pretty fast as well.
>>>> 
>>>> On Fri, Jun 24, 2011 at 5:54 PM, XiaoboGu <guxiaobo1982@gmail.com>
>>> wrote:
>>>> 
>>>>> 32Core, 256G RAM
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Ted Dunning [mailto:ted.dunning@gmail.com]
>>>>>> Sent: Saturday, June 25, 2011 1:37 AM
>>>>>> To: user@mahout.apache.org
>>>>>> Cc: dev@mahout.apache.org
>>>>>> Subject: Re: Can all the algorithms in Mahout be run locally without
>>> a
>>>>> Hadoop cluster.
>>>>>> 
>>>>>> Big iron is fine for some of the classifier stuff, but throughput
per
>>> $
>>>>> can
>>>>>> be higher for other algorithms with a cluster of smaller machines.
>>>>>> 
>>>>>> How big a machine are you talking about?  Even relatively small
>>> machines
>>>>> are
>>>>>> pretty massive any more.  8 core = 16 hyper-thread machines with
48GB
>>>>> seem
>>>>>> to be not even very impressive any more.
>>>>>> 
>>>>>> On Fri, Jun 24, 2011 at 1:47 AM, XiaoboGu <guxiaobo1982@gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>>> We will put a big SMP server to deploy Mahout.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> 
>>>>>>> Xiaobo Gu
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>> 
>>> 
> 

Mime
View raw message