jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Boston (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-3547) Improve ability of the OakDirectory to recover from unexpected file errors
Date Mon, 02 Nov 2015 14:37:27 GMT

    [ https://issues.apache.org/jira/browse/OAK-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985293#comment-14985293
] 

Ian Boston commented on OAK-3547:
---------------------------------

Version just pushed works in AEM6.2 MongoMK with each save of the directory list represented
as list of files with file size and sha1 as below

{code}
02.11.2015 14:29:00.344 *INFO* [aysnc-index-update-fulltext-async] org.apache.jackrabbit.oak.plugins.index.lucene.OakDirectory$GenerationalDirectoryListing
Saving Listing state to l_1446474540217 as _6.si,229,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.fdx,1143274,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.pos,2256835,da39a3ee5e6b4b0d3255bfef95601890afd80709;_23.cfs,1592,da39a3ee5e6b4b0d3255bfef95601890afd80709;_6.cfe,224,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.fdt,12261250,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.tip,156795,da39a3ee5e6b4b0d3255bfef95601890afd80709;segments_1x,154,da39a3ee5e6b4b0d3255bfef95601890afd80709;_23.si,232,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.tim,6207095,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.si,316,da39a3ee5e6b4b0d3255bfef95601890afd80709;_23.cfe,224,da39a3ee5e6b4b0d3255bfef95601890afd80709;segments.gen,20,da39a3ee5e6b4b0d3255bfef95601890afd80709;_6.cfs,5403594,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.fnm,61229,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.doc,2073360,da39a3ee5e6b4b0d3255bfef95601890afd80709;_6_1.del,51,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b_1.del,38,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.nvd,142931,da39a3ee5e6b4b0d3255bfef95601890afd80709;_b.nvm,46,da39a3ee5e6b4b0d3255bfef95601890afd80709;

{code}

Since the save happens every few seconds, still need to do something to prune the listings
and to delete files no longer referenced. Probably best done with code that runs every few
hours. 

Slightly concerned at the frequency of close or sync operations performed on the OakDirectory

> Improve ability of the OakDirectory to recover from unexpected file errors
> --------------------------------------------------------------------------
>
>                 Key: OAK-3547
>                 URL: https://issues.apache.org/jira/browse/OAK-3547
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: lucene
>    Affects Versions: 1.4
>            Reporter: Ian Boston
>
> Currently if the OakDirectory finds that a file is missing or in some way damaged, and
exception is thrown which impacts all queries using that index, at times making the index
unavailable. This improvement aims to make the OakDirectory recover to a previously ok state
by storing which files were involved in previous states, and giving the code some way of checking
if they are valid.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message