lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5636) SegmentCommitInfo continues to list unneeded gen'd files
Date Fri, 02 May 2014 17:53:21 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988005#comment-13988005
] 

Shai Erera commented on LUCENE-5636:
------------------------------------

I chatted with Robert about this. The current situation is that the old .fnm files continue
to be referenced even when not needed, however when the segment is merged, they go away (as
all gen'd files). Given that there's no way to solve it without breaking back-compat, unless
we introduce hacks such as checking for a ".fnm" suffix, we discussed  how to solve this "going
forward".

By "going forward" I mean to not change existing segments, but if they contain future updates,
write the new information in a better way. Perhaps old .fnm files will still be referenced
by those segments, until they're merged away, but new segments will fix that bug.

I think that this might be doable together with LUCENE-5618, by writing per-field gen'd DV
file, so I'll try to solve it there and if it works I'll resolve that issue as appropriate.

> SegmentCommitInfo continues to list unneeded gen'd files
> --------------------------------------------------------
>
>                 Key: LUCENE-5636
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5636
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>            Reporter: Shai Erera
>            Assignee: Shai Erera
>         Attachments: LUCENE-5636.patch
>
>
> I thought I handled it in LUCENE-5246, but turns out I didn't handle it fully. I'll upload
a patch which improves the test to expose the bug. I know where it is, but I'm not sure how
to fix it without breaking index back-compat. Can we do that on experimental features?
> The problem is that if you update different fields in different gens, the FieldInfos
files of older gens remain referenced (still!!). I open a new issue since LUCENE-5246 is already
resolved and released, so don't want to mess up our JIRA...
> The severity of the bug is that unneeded files are still referenced in the index. Everything
still works correctly, it's just that .fnm files are still there. But as I wrote, I'm still
not sure how to solve it without requiring apps that use dv updates to reindex.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message