jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Dürig (JIRA) <j...@apache.org>
Subject [jira] [Comment Edited] (OAK-3177) Compaction slow on repository with continuous writes
Date Wed, 05 Aug 2015 07:31:05 GMT

    [ https://issues.apache.org/jira/browse/OAK-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654944#comment-14654944
] 

Michael Dürig edited comment on OAK-3177 at 8/5/15 7:30 AM:
------------------------------------------------------------

Proposed patch introducing an onto state in the compactor for applying the differences between
the before and the after state. 

This patch seems to improve performance of subsequent compaction cycles. Will follow up with
numbers. 

[~alex.parvulescu], could you have a look? In particular at the questions in the TODOs in
the patch re. backup specific usages of the compactor. 


was (Author: mduerig):
Proposed patch introducing an onto state in the compactor for applying the differences between
the before and the after state. 

[~alex.parvulescu], could you have a look? In particular at the questions in the TODOs in
the patch re. backup specific usages of the compactor. 

> Compaction slow on repository with continuous writes
> ----------------------------------------------------
>
>                 Key: OAK-3177
>                 URL: https://issues.apache.org/jira/browse/OAK-3177
>             Project: Jackrabbit Oak
>          Issue Type: Sub-task
>          Components: segmentmk
>            Reporter: Michael Dürig
>            Assignee: Michael Dürig
>              Labels: compaction, gc
>             Fix For: 1.3.5
>
>         Attachments: OAK-3177.patch
>
>
> OAK-2734 introduced retry cycles and the option to force compaction when all cycles fail.
However OAK-2192 introduced a performance regression: each compaction cycle takes in the order
of the size of the repository to complete instead of in the order of the number of remaining
changes to compact. This is caused by comparing compacted with pre-compacted node states,
which is necessary to avoid mixed segments (aka OAK-2192). To fix the performance regression
I propose to pass the compactor an additional node state (the 'onto' state). The diff would
then be calculated across the pre compacted states, which performs in the order of number
of changes. The changes would then be applied to the 'onto' state, which is a compacted state
to avoid mixed segments. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message