hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: 2.0.4 beta vs 2.0.3
Date Mon, 06 May 2013 19:12:02 GMT

Looks like this bug reached a tipping point - multiple people reporting this after a long
long time ;)

+Vinod Kumar Vavilapalli

On May 6, 2013, at 8:09 AM, Jason Lowe wrote:

> This may be related to MAPREDUCE-5168 <https://issues.apache.org/jira/browse/MAPREDUCE-5168>.
There's a memory leak of sorts in the shuffle if many map outputs end up being merged from
disk.
> 
> Jason
> 
> On 05/04/2013 06:40 PM, Radim Kolar wrote:
>> After upgrade i am getting out of heap space during shuffle. I am using compressed
mapper outputs and 200 mb sort buffers. Was something important changed? like for example
allocating 200mb * number of fetchers now.
>> 
>> 2013-05-04 04:02:10,209 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception
running child : org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle
in fetcher#4
>>    at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:121)
>>    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:379)
>>    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
>>    at java.security.AccessController.doPrivileged(Native Method)
>>    at javax.security.auth.Subject.doAs(Subject.java:415)
>>    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478)
>>    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>>    at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:56)
>>    at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:46)
>>    at org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput.<init>(InMemoryMapOutput.java:63)
>>    at org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.unconditionalReserve(MergeManagerImpl.java:297)
>>    at org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.reserve(MergeManagerImpl.java:287)
>>    at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:360)
>>    at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:295)
>>    at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:154)
>> 
>> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message