jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Mueller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-2761) Persistent cache: add data in a different thread
Date Wed, 10 Feb 2016 07:43:18 GMT

    [ https://issues.apache.org/jira/browse/OAK-2761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15140469#comment-15140469
] 

Thomas Mueller commented on OAK-2761:
-------------------------------------

I didn't review the patch yet, but from reading the comments there is one problem:

> Now it puts items to the queue only when they are evicted from memCache or invalidated.

This is fine for the "standalone" persistent cache, that is, for a single instance. However,
it's not optimal for the broadcasting cache (where each cluster node is supposed to inform
other cluster nodes about additions to the cache). It would be good if the broadcasting cache
would receive entries when they are added to the (in-memory LIRS) cache, and the persistent
storage of the cache (MVStore currently) would get the entries when they are evicted. If it's
complicated to use different behavior for the broadcasting part, then I would prefer if we
can configure it (for example using a system property) so we can test it.

Another possible improvement is to only put those entries in the persistent storage if they
were accessed more than once. But this policy would need to be configurable so we can test
whether it helps (improve speed and reduce storage size).

> Persistent cache: add data in a different thread
> ------------------------------------------------
>
>                 Key: OAK-2761
>                 URL: https://issues.apache.org/jira/browse/OAK-2761
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: cache, core, mongomk
>            Reporter: Thomas Mueller
>            Assignee: Thomas Mueller
>              Labels: resilience
>             Fix For: 1.4
>
>         Attachments: AsyncCacheTest.patch, OAK-2761-1.2.patch, OAK-2761-trunk.patch
>
>
> The persistent cache usually stores data in a background thread, but sometimes (if a
lot of data is added quickly) the foreground thread is blocked.
> Even worse, switching the cache file can happen in a foreground thread, with the following
stack trace.
> {noformat}
> "127.0.0.1 [1428931262206] POST /bin/replicate.json HTTP/1.1" prio=5 tid=0x00007fe5df819800
nid=0x9907 runnable [0x0000000113fc4000]
>    java.lang.Thread.State: RUNNABLE
>         ...
> 	at org.h2.mvstore.MVStoreTool.compact(MVStoreTool.java:404)
> 	at org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache$1.closeStore(PersistentCache.java:213)
> 	- locked <0x0000000782483050> (a org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache$1)
> 	at org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache.switchGenerationIfNeeded(PersistentCache.java:350)
> 	- locked <0x0000000782455710> (a org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache)
> 	at org.apache.jackrabbit.oak.plugins.document.persistentCache.NodeCache.write(NodeCache.java:85)
> 	at org.apache.jackrabbit.oak.plugins.document.persistentCache.NodeCache.put(NodeCache.java:130)
> 	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.applyChanges(DocumentNodeStore.java:1060)
> 	at org.apache.jackrabbit.oak.plugins.document.Commit.applyToCache(Commit.java:599)
> 	at org.apache.jackrabbit.oak.plugins.document.CommitQueue.afterTrunkCommit(CommitQueue.java:127)
> 	- locked <0x0000000781890788> (a org.apache.jackrabbit.oak.plugins.document.CommitQueue)
> 	at org.apache.jackrabbit.oak.plugins.document.CommitQueue.done(CommitQueue.java:83)
> 	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.done(DocumentNodeStore.java:637)
> {noformat}
> To avoid blocking the foreground thread, one solution is to store all data in a separate
thread. If there is too much data added, then some of the data is not stored. If possible,
the data that was not referenced a lot, and / or old revisions of documents (if new revisions
are available).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message