mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tamas Jambor <jambo...@googlemail.com>
Subject mapreduce memory issues
Date Wed, 05 May 2010 18:03:21 GMT
Hi,

I came across a new problem with the mapreduce implementation. I am 
trying to optimize the cluster for this implemetation. But the problem 
is that in order to run 
RecommenderJob-UserVectorToCooccurrenceMapper-UserVectorToCooccurrenceReducer, 
I need to set -Xmx2048m, with a smaller value it fails the job. how come 
it needs so much memory? maybe there is a memory leak here? generally it 
is suggested to set -Xmx512m

the other problem setting it so high is that I have to reduce the number 
map/reduce jobs per node, otherwise the next job brings the whole 
cluster down.

Tamas


Mime
View raw message