mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed Elgohary <>
Subject OutOfMemoryError in MatrixMultiplicationJob
Date Sat, 15 Sep 2012 00:03:18 GMT

I was running mahout MatrixMultiplicationJob on Amazon EMR to multiply two
matrices of sizes (150k x 8.2m) and (100 x 150k ). My cluster consisted of
20 m1.large nodes. I am getting an OutOfMemoryError:

Error: java.lang.OutOfMemoryError: Java heap space
	at org.apache.mahout.math.RandomAccessSparseVector.setQuick(
	at org.apache.mahout.math.AbstractVector.assign(
	at org.apache.mahout.math.hadoop.MatrixMultiplicationJob$MatrixMultiplicationReducer.reduce(
	at org.apache.mahout.math.hadoop.MatrixMultiplicationJob$MatrixMultiplicationReducer.reduce(
	at org.apache.hadoop.mapred.Task$OldCombinerRunner.combine(
	at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$InMemFSMergeThread.doInMemMerge(
	at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$

I increased the heap size of the reducer task JVM up to 4GB. But, that did
not solve the problem.
I am looking for any suggestions to solve that problem.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message