lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From negrinv <>
Subject Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted
Date Fri, 01 Dec 2006 10:10:47 GMT

Nicolas Lalevée-2 wrote:
> Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit :
>> Thank you Robert for your commnets. I am inclined to agree with you, but
>> I
>> would like to establish first of all if simplicity of implementation is
>> the
>> overriding consideration. But before I dwell on that let me say that i
>> have
>> discovered that I am not a master of DIFF file creation with Eclipse. The
>> diff file attachement to my original posting is absurdly large and not
>> correct. I have therefore attached a zip file containing the complete
>> source code of the classes I modified. I leave it to others to extract
>> the
>> diffs properly.
>> Back to the issue. So far the implementation has not been difficult
>> considering that I knew nothing about Lucene internals before I started.
>> The reason is that Lucene is very well structured and the changes just
>> fitted nicely by adding some code in the right place with minimal changes
>> to the existing code. But I admit that the proposed implementation so far
>> is not complete and more work is required to overcome some of its
>> restrictions. While I like your idea I believe that it imposed too large
>> a
>> granularity on the encrypted data, all fields will all kinds of data will
>> be encrypted including  images and others which normally would be left
>> alone, thus adding to the performance penalty due to encryption.
> I don't agree with you here. In Lucene, you will encrypt the field data,
> the 
> field names, and the tokens : I would say that is represents at least 2/3
> of 
> the index size. Then, with the implementation you suggest, I think (sorry
> I 
> didn't took time to see you patch) that every time a lucene data need to
> be 
> read, it is decrypted each time. With an encrypted FS, your kernel will 
> maintain a cache in RAM for you, so it won't hurt so much.
> It needs some bench to see what is effectively the best, but I have doubt
> that 
> your solution will be faster.
> Nicolas.

Nicolas, I am all in favour of some tests to establish which solution is
best, but I have to say that I don't believe file system or directory
encryption in Lucene is really justified. Most operating system already
provide this feature, although they are system-wide or policy-based
solution, hence not always within individual user control. 
But if the issue is user control, then I believe Lucene should provide
maximum granularity when it comes to choice of data to encrypt.
The issue I believe is whether some form of encryption should be provided
within Lucene to enable application developers to create applications which
offer some data protection under user control, with a minimum of impact,
where by impact I mean both on peformance and workload either in Lucene code
or user code.
I cannot test the performance issues until there is an alternative solution
in place. If you have one and you can make it available I will be happy to
give it an impartial test.

View this message in context:
Sent from the Lucene - Java Developer mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message