jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Dürig (JIRA) <j...@apache.org>
Subject [jira] [Commented] (OAK-3348) Cross gc sessions might introduce references to pre-compacted segments
Date Tue, 15 Dec 2015 16:39:46 GMT

    [ https://issues.apache.org/jira/browse/OAK-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058292#comment-15058292

Michael Dürig commented on OAK-3348:

AFAICS there two things we need to fix to get rid of references to pre-compacted segments:
# Prevent segment node builders acquired before a compaction cycle to flush into compacted
segments. As such builders reference a base state that lives in pre-compacted segments, such
a flush pollutes compacted segments.
# When merging segment node builders that have been acquired before a compaction cycle prevent
them to link back to its pre-compacted base state. 

For 1) we can ensure that such node builders would write into their own segment. This should
be relatively easy by leveraging the borrow mechanism for {{SegmentBufferWriters}} introdues
in OAK-1828. 

For 2) we need to "compact" the base state of such builders after the fact. To make this efficient
(wrt. de-duplication) we need to pass the compaction map of the respective generation to the
compactor. (There is a slight chance that a builder is older than just a single gc generation,
in which case this approach is not correct. For the time being I consider this an edge case
and would just throw an {{CommitFailedException}} at this point).

> Cross gc sessions might introduce references to pre-compacted segments
> ----------------------------------------------------------------------
>                 Key: OAK-3348
>                 URL: https://issues.apache.org/jira/browse/OAK-3348
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: segmentmk
>            Reporter: Michael Dürig
>            Assignee: Michael Dürig
>              Labels: candidate_oak_1_0, candidate_oak_1_2, cleanup, compaction, gc
>             Fix For: 1.4
>         Attachments: OAK-3348-1.patch, OAK-3348-2.patch, OAK-3348.patch, cross-gc-refs.pdf,
> I suspect that certain write operations during compaction can cause references from compacted
segments to pre-compacted ones. This would effectively prevent the pre-compacted segments
from getting evicted in subsequent cleanup phases. 
> The scenario is as follows:
> * A session is opened and a lot of content is written to it such that the update limit
is exceeded. This causes the changes to be written to disk. 
> * Revision gc runs causing a new, compacted root node state to be written to disk.
> * The session saves its changes. This causes rebasing of its changes onto the current
root (the compacted one). At this point any node that has been added will be added again in
the sub-tree rooted at the current root. Such nodes however might have been written to disk
*before* revision gc ran and might thus be contained in pre-compacted segments. As I suspect
the node-add operation in the rebasing process *not* to create a deep copy of such nodes but
to rather create a *reference* to them, a reference to a pre-compacted segment is introduced
> Going forward we need to validate above hypothesis, assess its impact if necessary come
up with a solution.

This message was sent by Atlassian JIRA

View raw message