jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vikas Saurabh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-3976) journal should support large(r) entries
Date Mon, 12 Dec 2016 14:02:58 GMT

    [ https://issues.apache.org/jira/browse/OAK-3976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15741968#comment-15741968
] 

Vikas Saurabh commented on OAK-3976:
------------------------------------

Btw, ran our benchmark suite (except for Ordered\* ones) and ratio of {{N}} with/without the
patch above seems very similar... This is what I had run:
{noformat}
for JAR in oak-run-with-change.jar oak-run-without-change.jar;do
    echo "----- JarName $JAR -----"
    java -Xms4g -Xmx4g -jar $JAR benchmark Oak-MemoryNS AceCreationTest AddMemberTest AddMembersTest
BundlingNodeTest CompositeAuthorizationTest ConcurrentCreateNodesTest ConcurrentEveryoneACLTest
ConcurrentFileWriteTest ConcurrentHasPermissionTest ConcurrentHasPermissionTest2 ConcurrentReadAccessControlledTreeTest
ConcurrentReadAccessControlledTreeTest2 ConcurrentReadDeepTreeTest ConcurrentReadRandomNodeAndItsPropertiesTest
ConcurrentReadSinglePolicyTreeTest ConcurrentReadTest ConcurrentReadWriteTest ConcurrentTraversalTest
ConcurrentWriteACLTest ConcurrentWriteReadTest ConcurrentWriteTest ContinuousRevisionGCTest
CreateManyChildNodesTest CreateManyIndexedNodesTest CreateManyNodesTest CreateNodesBenchmark
CugOakTest CugTest DescendantSearchTest ExternalLoginTest FindAuthorizableWithScopeTest FlatTreeUpdateTest
FlatTreeWithAceForSamePrincipalTest FullTextSearchTest FullTextSolrSearchTest GetAuthorizableByIdTest
GetAuthorizableByPrincipalTest GetDeepNodeTest GetGroupPrincipalsTest GetNodeWithAdmin GetNodeWithAnonymous
GetPoliciesTest GetPrincipalTest HybridIndexTest IsDeclaredMemberTest IsMemberTest LinearReadEmpty
LinearReadFiles LinearReadNodes LoginGetRootLogoutTest LoginImpersonateTest LoginLogoutTest
LoginSystemTest LoginTest LoginWithMembersTest LoginWithMembershipTest LucenePropertyFTSeparated
LucenePropertyFullTextTest ManyNodes ManyUserReadTest MemberDeclaredMemberOf MemberIsDeclaredMember
MemberIsMember MemberMemberOf NamespaceRegistryTest NamespaceTest ObservationTest PersistentCacheTest
ReadDeepTreeTest ReadPropertyTest RemoveMemberTest RemoveMembersTest ReplicaCrashResilienceTest
RepositoryGrowthTest RevisionGCTest SQL2DescendantSearchTest SQL2SearchTest SequentialCreateNodesTest
SetMultiPropertyTest SetPropertyTest SimpleSearchTest SmallFileReadTest SmallFileWriteTest
SyncAllExternalUsersTest SyncExternalUsersTest TransientManyChildNodesTest UUIDLookupTest
UniformReadEmpty UniformReadFiles UniformReadNodes UpdateManyChildNodesTest
done
{noformat}

> journal should support large(r) entries
> ---------------------------------------
>
>                 Key: OAK-3976
>                 URL: https://issues.apache.org/jira/browse/OAK-3976
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: documentmk
>    Affects Versions: 1.3.14
>            Reporter: Stefan Egli
>            Assignee: Vikas Saurabh
>             Fix For: 1.6, 1.5.16
>
>
> Journal entries are created in the background write. Normally this happens every second.
If for some reason there is a large delay between two background writes, the number of pending
changes can also accumulate. Which can result in (arbitrary) large single journal entries
(ie with large {{_c}} property).
> This can cause multiple problems down the road:
> * journal gc at this point loads 450 entries - and if some are large this can result
in a very large memory consumption during gc (which can cause severe stability problems for
the VM, if not OOM etc). This should be fixed with OAK-3001 (where we only get the id, thus
do not care how big {{_c}} is)
> * before OAK-3001 is done (which is currently scheduled after 1.4) what we can do is
reduce the delete batch size (OAK-3975)
> * background reads however also read the journal entries and even if OAK-3001/OAK-3975
are implemented the background read can still cause large memory consumption. So we need to
improve this one way or another.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message