flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nico Kruber (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-10317) Configure Metaspace size by default
Date Tue, 30 Apr 2019 12:52:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16725939#comment-16725939
] 

Nico Kruber edited comment on FLINK-10317 at 4/30/19 12:51 PM:
---------------------------------------------------------------

-Actually limiting metaspace will also trigger its garbage collection (without limit it won't
do GC for the metaspace!).-(GC is done once the initial metaspace size is hit and from then
on the GC decides - see https://blogs.oracle.com/poonam/about-g1-garbage-collector,-permanent-generation-and-metaspace)
Did you test this out with the metaspace limit?

If setting metaspace correctly (please verify in the TM logs) does not help, then your issue
should be continued in a separate thread like FLINK-9080 or FLINK-11205.


was (Author: nicok):
Actually limiting metaspace will also trigger its garbage collection (without limit it won't
do GC for the metaspace!).
Did you test this out with the metaspace limit?

If setting metaspace correctly (please verify in the TM logs) does not help, then your issue
should be continued in a separate thread like FLINK-9080 or FLINK-11205.

> Configure Metaspace size by default
> -----------------------------------
>
>                 Key: FLINK-10317
>                 URL: https://issues.apache.org/jira/browse/FLINK-10317
>             Project: Flink
>          Issue Type: Bug
>          Components: Deployment / Scripts
>    Affects Versions: 1.5.3, 1.6.0, 1.7.0
>            Reporter: Stephan Ewen
>            Priority: Major
>             Fix For: 1.7.3, 1.6.5
>
>         Attachments: Screen Shot 2019-01-07 at 10.08.39.png, Screen Shot 2019-01-07 at
10.08.47.png, Screenshot 2018-12-18 at 12.14.11.png
>
>
> We should set the size of the JVM Metaspace to a sane default, like  {{-XX:MaxMetaspaceSize=256m}}.
> If not set, the JVM offheap memory will grow indefinitely with repeated classloading
and Jitting, eventually exceeding allowed memory on docker/yarn or similar setups.
> It is hard to come up with a good default, however, I believe the error messages one
gets when metaspace is too small are easy to understand (and easy to take action), while it
is very hard to figure out why the memory footprint keeps growing steadily and infinitely.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message