tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajesh Balamohan <rbalamo...@apache.org>
Subject Re: Difference between COMMITTED_HEAP_BYTES and PHYSICAL_MEMORY_BYTES
Date Sun, 18 Dec 2016 03:09:29 GMT
Hi,

PHYSICAL_MEMORY_BYTES represents the cumulative RSS memory of the process
tree and COMMITTED_HEAP_BYTES is the current total committed heap space
usage of this JVM. COMMITTED_HEAP_BYTES is computed via
Runtime.getRuntime.totalMemory().

But when TezMxBeanResourceCalculator is used which is the default in recent
tez versions, PHYSICAL_MEMORY_BYTES is also computed via
getRuntime().totalMemory(). Hence in your queries, you see both the values
representing the same value.

https://issues.apache.org/jira/browse/TEZ-1698 has more details on this.

~Rajesh.B

On Sun, Dec 18, 2016 at 12:53 AM, Ranjan Banerjee <rabaner@microsoft.com>
wrote:

> Hi,
>    Could someone please explain the difference between
> PHYSICAL_MEMORY_BYTES and COMMITTED_HEAP_BYTES.
>     From my understanding COMMITTED_HEAP_BYTES is the xmx value in the tez
> container and PHYSICAL_MEMORY_BYTES is the actual memory used by the task.
>
> But in my sample queries I see both of them to be the exact value, does
> this mean that my task is using 100% of the memory assigned? Some
> clarification would be greatly appreciated!
>
> Thanks
> Ranjan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message