jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Parvulescu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (OAK-2192) Concurrent commit during compaction results in mixed segments
Date Sun, 09 Nov 2014 12:41:34 GMT

     [ https://issues.apache.org/jira/browse/OAK-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alex Parvulescu updated OAK-2192:
---------------------------------
    Attachment: OAK-2192-v8.patch

v8 of this patch. I see no more SNFEs, but there are quite some changes included: 
 - really add everything to the compaction map (all nodes, all binaries), 
 - segmentid equals check needs to change, the old impl cannot work given the cleanup approaches
we have now in place
 - tarreader cleanup will now try to take into account the compaction map in the case of binary
segments. this helps with currently running diffs over the old states.
To explain this a bit better: the compaction map is quite efficient in linking old states
to the new versions, but the only window of trouble we have now are already running diffs,
where loading the current node state triggers loading a segment that may be already gone.
this is where you can really see the difference between the 2 available cleanup strategies:
'all' will cleanup everything but gives no fallback for this scenario, 'timestamp' will allow
running diffs to catchup at the cost of having to keep some garbage around for a longer time.
the change on v8 is that now we take into account bulk segments already compacted and skip
them to make this process more efficient.
 - switched the "pause" flag to false to allow compaction to run.


TODO we possibly need to remove the SNFE logs in the SegmentTracker, cleanup.



> Concurrent commit during compaction results in mixed segments
> -------------------------------------------------------------
>
>                 Key: OAK-2192
>                 URL: https://issues.apache.org/jira/browse/OAK-2192
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: core
>            Reporter: Michael Dürig
>            Assignee: Michael Dürig
>              Labels: compaction, gc
>         Attachments: OAK-2192-2.patch, OAK-2192-poc-fix.patch, OAK-2192-possible-test.patch,
OAK-2192-v2.patch, OAK-2192-v3.patch, OAK-2192-v4.patch, OAK-2192-v5.patch, OAK-2192-v6.patch,
OAK-2192-v7.patch, OAK-2192-v8.patch, OAK-2192.patch
>
>
> Changes that are committed during a segment store compaction run will be compacted on
top of the already compacted changes. However the compactor uses the wrong before state in
this case. Instead of compacting against the compacted before state it uses the un-compacted
before state. The resulting state will thus contain references to un-compacted state, making
those not eligible for later clean up. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message