lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Engels <>
Subject Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted
Date Fri, 01 Dec 2006 17:36:00 GMT
Not if running under OSX with encrypted swap turned on ! :)

-----Original Message-----
>From: Nicolas Lalevée <>
>Sent: Dec 1, 2006 4:49 AM
>Subject: Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted
>Le Vendredi 1 Décembre 2006 11:10, negrinv a écrit :
>> Nicolas Lalevée-2 wrote:
>> > Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit :
>> >> Thank you Robert for your commnets. I am inclined to agree with you, but
>> >> I
>> >> would like to establish first of all if simplicity of implementation is
>> >> the
>> >> overriding consideration. But before I dwell on that let me say that i
>> >> have
>> >> discovered that I am not a master of DIFF file creation with Eclipse.
>> >> The diff file attachement to my original posting is absurdly large and
>> >> not correct. I have therefore attached a zip file containing the
>> >> complete source code of the classes I modified. I leave it to others to
>> >> extract the
>> >> diffs properly.
>> >> Back to the issue. So far the implementation has not been difficult
>> >> considering that I knew nothing about Lucene internals before I started.
>> >> The reason is that Lucene is very well structured and the changes just
>> >> fitted nicely by adding some code in the right place with minimal
>> >> changes to the existing code. But I admit that the proposed
>> >> implementation so far is not complete and more work is required to
>> >> overcome some of its restrictions. While I like your idea I believe that
>> >> it imposed too large a
>> >> granularity on the encrypted data, all fields will all kinds of data
>> >> will be encrypted including  images and others which normally would be
>> >> left alone, thus adding to the performance penalty due to encryption.
>> >
>> > I don't agree with you here. In Lucene, you will encrypt the field data,
>> > the
>> > field names, and the tokens : I would say that is represents at least 2/3
>> > of
>> > the index size. Then, with the implementation you suggest, I think (sorry
>> > I
>> > didn't took time to see you patch) that every time a lucene data need to
>> > be
>> > read, it is decrypted each time. With an encrypted FS, your kernel will
>> > maintain a cache in RAM for you, so it won't hurt so much.
>> > It needs some bench to see what is effectively the best, but I have doubt
>> > that
>> > your solution will be faster.
>> >
>> > Nicolas.
>> Nicolas, I am all in favour of some tests to establish which solution is
>> best, but I have to say that I don't believe file system or directory
>> encryption in Lucene is really justified. Most operating system already
>> provide this feature, although they are system-wide or policy-based
>> solution, hence not always within individual user control.
>> But if the issue is user control, then I believe Lucene should provide
>> maximum granularity when it comes to choice of data to encrypt.
>> The issue I believe is whether some form of encryption should be provided
>> within Lucene to enable application developers to create applications which
>> offer some data protection under user control, with a minimum of impact,
>> where by impact I mean both on peformance and workload either in Lucene
>> code or user code.
>In fact you mean a user that has no control of it's machine, and that cannot 
>encrypt his partition. Here you will have the issue with the swap : Lucene 
>will decrypt the data in RAM, that can possibly pushed on the swap... I know 
>this is extreme, but it's a security hole.
>Nicolas LALEVÉE
>Solutions & Technologies
>Tel : +33 (0)5 61 00 52 90
>Fax : +33 (0)5 61 00 51 46
>To unsubscribe, e-mail:
>For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message