cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Williams (JIRA)" <>
Subject [jira] Created: (CASSANDRA-1014) GC storming, possible memory leak
Date Thu, 22 Apr 2010 16:19:51 GMT
GC storming, possible memory leak

                 Key: CASSANDRA-1014
             Project: Cassandra
          Issue Type: Bug
    Affects Versions: 0.6.1
         Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11, mixed

            Reporter: Brandon Williams
         Attachments: 724-0001.png

There appears to be a GC issue due to memory pressure in the 0.6 branch.  You can see this
by starting the server and performing many inserts.  Quickly the jvm will consume most of
its heap, and pauses for stop-the-world GC will begin.  With verbose GC turned on, this can
be observed as follows:

[GC [ParNew (promotion failed): 79703K->79703K(84544K), 0.0622980 secs][CMS[CMS-concurrent-mark:
3.678/5.031 secs] [Times: user=10.35 sys=4.22, real=5.03 secs]
 (concurrent mode failure): 944529K->492222K(963392K), 2.8264480 secs] 990745K->492222K(1047936K),
2.8890500 secs] [Times: user=2.90 sys=0.04, real=2.90 secs]

After enough inserts (around 75-100 million) the server will GC storm and then OOM.

jbellis and I narrowed this down to patch 0001 in CASSANDRA-724.  Switching LBQ with ABQ made
no difference, however using batch mode instead of periodic for the commitlog does prevent
the issue from occurring.  The attached screenshot show the heap usage in jconsole first when
the issue is exhibiting, a restart, and then the same amount of inserts when it does not.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message