cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonas Borgström (JIRA) <>
Subject [jira] [Commented] (CASSANDRA-15006) Possible java.nio.DirectByteBuffer leak
Date Fri, 01 Mar 2019 10:23:00 GMT


Jonas Borgström commented on CASSANDRA-15006:

[~benedict] I've now emailed you links to the heap dumps I have.

> Possible java.nio.DirectByteBuffer leak
> ---------------------------------------
>                 Key: CASSANDRA-15006
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: cassandra: 3.11.3
> jre: openjdk version "1.8.0_181"
> heap size: 2GB
> memory limit: 3GB (cgroup)
> I started one of the nodes with "-Djdk.nio.maxCachedBufferSize=262144" but that did not
seem to make any difference.
>            Reporter: Jonas Borgström
>            Priority: Major
>         Attachments: CASSANDRA-15006-reference-chains.png, Screenshot_2019-02-04 Grafana
- Cassandra.png, Screenshot_2019-02-14 Grafana - Cassandra(1).png, Screenshot_2019-02-14 Grafana
- Cassandra.png, Screenshot_2019-02-15 Grafana - Cassandra.png, Screenshot_2019-02-22 Grafana
- Cassandra.png, Screenshot_2019-02-25 Grafana - Cassandra.png, cassandra.yaml, cmdline.txt
> While testing a 3 node 3.11.3 cluster I noticed that the nodes were suddenly killed by
the Linux OOM killer after running without issues for 4-5 weeks.
> After enabling more metrics and leaving the nodes running for 12 days it sure looks like
> "java.nio:type=BufferPool,name=direct" Mbean shows a very linear growth (approx 15MiB/24h,
see attached screenshot). Is this expected to keep growing linearly after 12 days with a constant
> In my setup the growth/leak is about 15MiB/day so I guess in most setups it would take
quite a few days until it becomes noticeable. I'm able to see the same type of slow growth
in other production clusters even though the graph data is more noisy.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message