jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chetan Mehrotra (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-2557) VersionGC uses way too much memory if there is a large pile of garbage
Date Thu, 12 Mar 2015 11:20:39 GMT

    [ https://issues.apache.org/jira/browse/OAK-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358517#comment-14358517
] 

Chetan Mehrotra commented on OAK-2557:
--------------------------------------

Thanks for the detailed review [~mreutegg]. Attached is the [updated patch|^OAK-2557-3.patch]

bq. Maybe even promote NodeDocIdCollector to a top level class to avoid breaking encapsulation?

That makes more sense as its a generic class not bound to Document logic. I have refactored
and moved this class to oak-commons under sort package. [~tmueller] Can you have a quick look
as with this it becomes part of Oak API

bq. NodeDocIdCollector.sort() uses a hard coded Comparator when sorting in memory instead
of the instance passed in the constructor.

Fixed that

bq. NodeDocIdCollector.flushToFile() uses PrintWriter.println(). This method does not throw
an IOException when the write fails. I think it would be better to use BufferedWriter directly.

Did not knew that. Refactored to use {{BufferedWriter}}

bq. VersionGCState.close() deletes the the directory before resources are closed. I think
this will fail on Windows based machines.

Thanks for catching that. Would have missed this completely!

> VersionGC uses way too much memory if there is a large pile of garbage
> ----------------------------------------------------------------------
>
>                 Key: OAK-2557
>                 URL: https://issues.apache.org/jira/browse/OAK-2557
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: core, mongomk
>    Affects Versions: 1.0.11
>            Reporter: Stefan Egli
>            Assignee: Chetan Mehrotra
>            Priority: Blocker
>             Fix For: 1.0.13, 1.2
>
>         Attachments: OAK-2557-2.patch, OAK-2557-3.patch, OAK-2557.patch
>
>
> It has been noticed that on a system where revision-gc (VersionGarbageCollector of mongomk)
did not run for a few days (due to not interfering with some tests/large bulk operations)
that there was such a large pile of garbage accumulating, that the following code
> {code}
> VersionGarbageCollector.collectDeletedDocuments
> {code}
> in the for loop, creates such a large list of NodeDocuments to delete (docIdsToDelete)
that it uses up too much memory, causing the JVM's GC to constantly spin in Full-GCs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message