hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: hama processes
Date Fri, 28 Aug 2015 02:04:58 GMT
> for 3 Groom child process + 2GB for Ubuntu OS). Is this correct
> understanding ?

and,

> on a big dataset. I think these exceptions have something to do with Ubuntu
> OS killing the hama process due to lack of memory. So, I was curious about

Yes, you're right.

Each task processor has two message manager, one for outgoing and one
for incoming. All these are handled in memory, so it sometimes
requires large memory space. To solve the OutOfMemory issue, you
should scale-out your cluster by increasing the number of nodes and
job tasks, or optimize your algorithm. Another option is
disk-spillable message manager. This is not supported yet.

On Fri, Aug 28, 2015 at 10:45 AM, Behroz Sikander <behroz89@gmail.com> wrote:
> Hi,
> Yes. According to hama-default.xml, each machine will open 3 process with
> 2GB memory each. This means that my VMs need atleast 8GB memory (2GB each
> for 3 Groom child process + 2GB for Ubuntu OS). Is this correct
> understanding ?
>
> I recently ran into the following exceptions when I was trying to run hama
> on a big dataset. I think these exceptions have something to do with Ubuntu
> OS killing the hama process due to lack of memory. So, I was curious about
> my configurations.
> 'BSP task process exit with nonzero status of 137.'
> 'BSP task process exit with nonzero status of 1'
>
>
>
> Regards,
> Behroz
>
> On Fri, Aug 28, 2015 at 3:04 AM, Edward J. Yoon <edwardyoon@apache.org>
> wrote:
>
>> Hi,
>>
>> You can change the max tasks per node by setting below property in
>> hama-site.xml. :-)
>>
>>   <property>
>>     <name>bsp.tasks.maximum</name>
>>     <value>3</value>
>>     <description>The maximum number of BSP tasks that will be run
>> simultaneously
>>     by a groom server.</description>
>>   </property>
>>
>>
>> On Fri, Aug 28, 2015 at 5:18 AM, Behroz Sikander <behroz89@gmail.com>
>> wrote:
>> > Hi,
>> > Recently, I noticed that my hama deployment is only opening 3 processes
>> per
>> > machine. This is because of the configuration settings in the default
>> hama
>> > file.
>> >
>> > My questions is why 3 and why not 5 or 7 ? What criteria's should be
>> > considered if I want to increase the value ?
>> >
>> > Regards,
>> > Behroz
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>>



-- 
Best Regards, Edward J. Yoon

Mime
View raw message