jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Mueller (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (OAK-3007) SegmentStore cache does not take "string" map into account
Date Thu, 09 Jul 2015 12:13:04 GMT

     [ https://issues.apache.org/jira/browse/OAK-3007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Thomas Mueller updated OAK-3007:
    Attachment: OAK-3007-3.patch

OAK-3007-3.patch: new patch with slightly changed API to simplify testing (making the StringCache
independent on the Segment class).

I run some benchmarks and got slightly lower numbers, but the numbers varied quite a bit so
I'm not sure if this is really related to my changes. Profiling does not show this code anywhere
in the critical path. However, Segment.readTemplate is in the critical path, and I wonder
if we should use a faster array-based cache there as well (similar to the StringCache.FastCache),
in addition to the (unlimited size) concurrent hash map. I guess there are not that many templates
typically so that this is not such a big risk to run out of memory with many templates, except
in some exotic use cases (randomly generated property names for example).

> SegmentStore cache does not take "string" map into account
> ----------------------------------------------------------
>                 Key: OAK-3007
>                 URL: https://issues.apache.org/jira/browse/OAK-3007
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: segmentmk
>            Reporter: Thomas Mueller
>             Fix For: 1.3.3
>         Attachments: OAK-3007-2.patch, OAK-3007-3.patch, OAK-3007.patch
> The SegmentStore cache size calculation ignores the size of the field Segment.string
(a concurrent hash map). It looks like a regular segment in a memory mapped file has the size
1024, no matter how many strings are loaded in memory. This can lead to out of memory. There
seems to be no way to limit (configure) the amount of memory used by strings. In one example,
100'000 segments are loaded in memory, and 5 GB are used for Strings in that map.
> We need a way to configure the amount of memory used for that. This seems to be basically
a cache. OAK-2688 does this, but it would be better to have one cache with a configurable
size limit.

This message was sent by Atlassian JIRA

View raw message