hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maysam Yabandeh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6118) Uber-mode decision does not consider -Xmx
Date Thu, 02 Oct 2014 21:57:34 GMT
Maysam Yabandeh created MAPREDUCE-6118:

             Summary: Uber-mode decision does not consider -Xmx
                 Key: MAPREDUCE-6118
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6118
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Maysam Yabandeh
            Priority: Minor

Current the decision on using uber-mode is based on the mapper container size and the AM container
size. However the actual memory at AM is limited by -Xmx options passed via yarn.app.mapreduce.am.command-opts.

For example: AM memory: 4G, yarn.app.mapreduce.am.command-opts=-Xmx2048GB, mapper memory:
3GB. Here the uber job could face OOM whereas a non-uber execution would not.

We faced this problem recently and forced to disable uber-mode to circumvent the problem.
One idea, although a bit ugly, is to parse jvm opts, extract Xmx, and take it into account
for uber-mode decision.

This message was sent by Atlassian JIRA

View raw message