samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Shaw <jor...@pubnub.com>
Subject Java Opts and Max Heap
Date Tue, 10 Mar 2015 08:13:53 GMT
Hey Everyone,
This I have a question somewhat related to SAMZA-109 and this line in
run-class.sh:
# Check if a max-heap size is specified. If not - set a 768M heap [[
$JAVA_OPTS != *-Xmx* ]] && JAVA_OPTS="$JAVA_OPTS -Xmx768M"

If I were to set the container.memory.mb for yarn to 4GB (
yarn.container.memory.mb = 4096) the above JAVA_OPTS check would cause this
to be ignored right? I can understand why preventing the Heap from going
crazy is important in these long running jobs but to me this might cause
some confusion especially when trying to debug a thrashing GC and Java
isn't committing the memory amount set in yarn.container.memory.mb.

I was trying to follow why this was decided on in SAMZA-109 but nothing
stuck out to me. Would it make more sense to just not specifying a max heap
and letting Yarn kill the job if it goes over it's specified allotment
and/or let the user explicitly set these opts?  Thanks!

- Jordan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message