jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomek Rękawek (JIRA) <j...@apache.org>
Subject [jira] [Comment Edited] (OAK-8185) Improve CompositeNodeStore performance
Date Tue, 02 Apr 2019 12:03:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807672#comment-16807672
] 

Tomek Rękawek edited comment on OAK-8185 at 4/2/19 12:02 PM:
-------------------------------------------------------------

Let me elaborate a bit on the way how the Composite Node Store works with regards to the mount
support. We have a single writeable mount {{/}} and a number of read-only mounts. For the
production purposes we usually have a single read-only mount, called {{libs}}. This mount
is available under a few paths:

* /libs
* /apps
* /jcr:system/rep:permissionStore/oak:mount-libs-crx.default

Apart from that there's something called "path-fragments", which allows to have dynamically
mounted paths. If, in the global mount, we have a node named {{oak:mount-libs-...}}, its subtree
will be also taken from the {{libs}} mount.

Now, in order to make the composite node store performant, we need to minimize the number
of places where we have the CompositeNodeState/Builder wrappers. For instance, we need to
have such a wrapper for the {{/}}, because calling {{getChild("libs")}} should give us a node
from one node store while calling {{getChild("content")}} should redirect us to another. The
path fragments makes this task a bit difficult, because the {{oak:mount-...}} may appear anywhere.
In order to mitigate this issue, the mount configuration has one more property: {{pathsSupportingFragments}}.
This way we can define the subtrees supporting potential dynamic mounts. The {{pathsSupportingFragments}}
supports pattern, so we can also define the depth of this tree. For instance, the for the
production we use:

{noformat}
/oak:index/*$
{noformat}

This way, the pathSupportingFragments are only supported under the direct children of the
/oak:index. See OAK-6585 for more info on the patterns.

With this setup, we were able to limit the instances of CompositeNodeState/Builder wrappers
only to following paths:

* /
* /jcr:system
* /jcr:system/rep:permissionStore
* /oak:index
* /oak:index/[index-name]

Now, about the benchmarks. Since the Composite Node Store requires a few repositories that
are joined together, it's quite hard to prepare a fixture that'll match the production env
with all the benchmarks. The current Oak-Composite-Store fixture mounts a few read-only repositories
under a few paths using the pattern {{mount-%d-path-%d}}. However, they are not used for reading.
The {{pathsSupportingFragments}} is set to {{/}}, therefore the Composite Node Store doesn't
use the skip-to-native-store optimization described above. After switching the {{pathSupportingFragments}}
to something more restrictive, like {{/oak:index/*$}}, the only composite-wrapped node will
be {{/}}. See  [^OAK-8185-update-fixture.patch]. The numbers looks as follows:

{noformat}
Apache Jackrabbit Oak 1.12-SNAPSHOT
# GetDeepNodeTest                  C     min     10%     50%     90%     max       N
Oak-Segment-Tar                    1      49      51      54      59      76    1095
Oak-Composite-Store                1      57      59      63      69     105     936
{noformat}

I'm open to suggestions on how to reflect the Composite Node Store performance better in the
oak-benchmarks. I feel that right now (before or after applying the patch) the numbers are
not really relevant for the real-life usage.


was (Author: tomek.rekawek):
Let me elaborate a bit on the way how the Composite Node Store works with regards to the mount
support. We have a single writeable mount {{/}} and a number of read-only mounts. For the
production purposes we usually have a single read-only mount, called {{libs}}. This mount
is available under a few paths:

* /libs
* /apps
* /jcr:system/rep:permissionStore/oak:mount-libs-crx.default

Apart from that there's something called "path-fragments", which allows to have dynamically
mounted paths. If, in the global mount, we have a node named {{oak:mount-libs-...}}, its subtree
will be also taken from the {{libs}} mount.

Now, in order to make the composite node store performant, we need to minimize the number
of places where we have the CompositeNodeState/Builder wrappers. For instance, we need to
have such a wrapper for the {{/}}, because calling {{getChild("libs")}} should give us a node
from one node store while calling {{getChild("content")}} should redirect us to another. The
path fragments makes this task a bit difficult, because the {{oak:mount-...}} may appear anywhere.
In order to mitigate this issue, the mount configuration has one more property: {{pathsSupportingFragments}}.
This way we can define the subtrees supporting potential dynamic mounts. The {{pathsSupportingFragments}}
supports pattern, so we can also define the depth of this tree. For instance, the for the
production we use:

{noformat}
/oak:index/*$
{noformat}

This way, the pathSupportingFragments are only supported under the direct children of the
/oak:index. See OAK-6585 for more info on the patterns.

With this setup, we were able to limit the instances of CompositeNodeState/Builder wrappers
only to following paths:

* (will complete this list in a few minutes)

Now, about the benchmarks. Since the Composite Node Store requires a few repositories that
are joined together, it's quite hard to prepare a fixture that'll match the production env
with all the benchmarks. The current Oak-Composite-Store fixture mounts a few read-only repositories
under a few paths using the pattern {{mount-%d-path-%d}}. However, they are not used for reading.
The {{pathsSupportingFragments}} is set to {{/}}, therefore the Composite Node Store doesn't
use the skip-to-native-store optimization described above. After switching the {{pathSupportingFragments}}
to something more restrictive, like {{/oak:index/*$}}, the only composite-wrapped node will
be {{/}}. See  [^OAK-8185-update-fixture.patch]. The numbers looks as follows:

{noformat}
Apache Jackrabbit Oak 1.12-SNAPSHOT
# GetDeepNodeTest                  C     min     10%     50%     90%     max       N
Oak-Segment-Tar                    1      49      51      54      59      76    1095
Oak-Composite-Store                1      57      59      63      69     105     936
{noformat}

I'm open to suggestions on how to reflect the Composite Node Store performance better in the
oak-benchmarks. I feel that right now (before or after applying the patch) the numbers are
not really relevant for the real-life usage.

> Improve CompositeNodeStore performance
> --------------------------------------
>
>                 Key: OAK-8185
>                 URL: https://issues.apache.org/jira/browse/OAK-8185
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: store-composite
>            Reporter: Marcel Reutegger
>            Assignee: Tomek Rękawek
>            Priority: Minor
>         Attachments: OAK-8185-update-fixture.patch, composite-node-builder.txt
>
>
> While working on OAK-8141 I noticed the benchmark numbers for GetDeepNodeTest on Oak-Composite-Store
are rather low compared to Oak-Segment-Tar.
> {noformat}
> Apache Jackrabbit Oak 1.12-SNAPSHOT
> # GetDeepNodeTest                  C     min     10%     50%     90%     max       N

> Oak-Segment-Tar                    1      35      37      39      41      64    1524
> Oak-Composite-Store                1     203     204     208     214     236     288
> {noformat}
> In an offline conversation [~tomek.rekawek] mentioned the overhead shouldn't be that
big because the implementation should switch to the non-composite implementation as soon as
the read operation traverses into the global/writable node store. It seems however, this is
not the case  when running GetDeepNodeTest. So, this may well be a bug and not an improvement,
as filed at the moment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message