lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From negrinv <>
Subject Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted
Date Tue, 05 Dec 2006 23:08:02 GMT

there is absolutely no suggestion to make any changes to the index format.
the index format would not change, whether you use encryption or not. 

Chuck Williams-2 wrote:
> Mike Klaas wrote on 12/05/2006 11:38 AM:
>> On 12/5/06, negrinv <> wrote:
>>> Chris Hostetter wrote:
>>> > If the code was not already in the core, and someone asked about
>>> adding it
>>> > I would argue against doing so on the grounds that some helpfull
>>> utility
>>> > methods (possibly in a contrib) would be just as usefull, and would
>>> have
>>> > no performance cost for people who don't care about compression.
>>> >
>>> Perhaps, if you look at compression on its own, but once you see
>>> compression
>>> in the context of all the other field options it makes sense to have it
>>> added to Lucene, it's about having everything in one place for ease of
>>> implementation that offsets the performance issue, in my opinion.
>> Note that built-in compression is deprecated, for similar reasons as
>> are being given for the encrypted fields.
> Built-in compression is also memory-hungry and slow due to the copying
> it does.  External compression is much faster, especially if you extend
> Field binary values to support a binary length parameter (which I
> submitted a patch for a long time ago).
> Here is another argument against adding Field encryption to the lucene
> core.  Changes in index format make life complex for any implementations
> that deal with index files directly.  There are a number of Lucene
> sister projects that do this, plus a number of applications.
> I have a fast bulk updater that directly manipulates index files and am
> busy upgrading it right now to the 2.1 index format with lockless
> commits (which is not fully documented in the new index file formats, by
> the way, e.g. the <segment>N.sM separate norm files are missing).  It's
> a pain.  In general, I think changes to Lucene index format should only
> be driven by compelling benefits.  Moving encryption from external to
> internal to get a minor application simplification is not sufficiently
> compelling to me.
> Chuck
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View this message in context:
Sent from the Lucene - Java Developer mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message