lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3201) improved compound file handling
Date Mon, 20 Jun 2011 19:00:47 GMT


Simon Willnauer commented on LUCENE-3201:

this seems ready to commit... I think we should get that in so I can take it further on LUCENE-3218

Robert is it ok for you if I commit this or are you gonig to do it?


> improved compound file handling
> -------------------------------
>                 Key: LUCENE-3201
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Robert Muir
>             Fix For: 3.3, 4.0
>         Attachments: LUCENE-3201.patch, LUCENE-3201.patch
> Currently CompoundFileReader could use some improvements, i see the following problems
> * its CSIndexInput extends bufferedindexinput, which is stupid for directories like mmap.
> * it seeks on every readInternal
> * its not possible for a directory to override or improve the handling of compound files.
> for example: it seems if you were impl'ing this thing from scratch, you would just wrap
the II directly (not extend BufferedIndexInput,
> and add compound file offset X to seek() calls, and override length(). But of course,
then you couldnt throw read past EOF always when you should,
> as a user could read into the next file and be left unaware.
> however, some directories could handle this better. for example MMapDirectory could return
an indexinput that simply mmaps the 'slice' of the CFS file.
> its underlying bytebuffer etc naturally does bounds checks already etc, so it wouldnt
need to be buffered, not even needing to add any offsets to seek(),
> as its position would just work.
> So I think we should try to refactor this so that a Directory can customize how compound
files are handled, the simplest 
> case for the least code change would be to add this to
> {code}
>   public Directory openCompoundInput(String filename) {
>     return new CompoundFileReader(this, filename);
>   }
> {code}
> Because most code depends upon the fact compound files are implemented as a Directory
and transparent. at least then a subclass could override...
> but the 'recursion' is a little ugly... we could still label it expert+internal+experimental
or whatever.

This message is automatically generated by JIRA.
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message