lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Lalevée <>
Subject Re: Attached proposed modifications to Lucene 2.0 to support Field.Store.Encrypted
Date Fri, 01 Dec 2006 10:49:11 GMT
Le Vendredi 1 Décembre 2006 11:10, negrinv a écrit :
> Nicolas Lalevée-2 wrote:
> > Le Vendredi 1 Décembre 2006 01:33, negrinv a écrit :
> >> Thank you Robert for your commnets. I am inclined to agree with you, but
> >> I
> >> would like to establish first of all if simplicity of implementation is
> >> the
> >> overriding consideration. But before I dwell on that let me say that i
> >> have
> >> discovered that I am not a master of DIFF file creation with Eclipse.
> >> The diff file attachement to my original posting is absurdly large and
> >> not correct. I have therefore attached a zip file containing the
> >> complete source code of the classes I modified. I leave it to others to
> >> extract the
> >> diffs properly.
> >> Back to the issue. So far the implementation has not been difficult
> >> considering that I knew nothing about Lucene internals before I started.
> >> The reason is that Lucene is very well structured and the changes just
> >> fitted nicely by adding some code in the right place with minimal
> >> changes to the existing code. But I admit that the proposed
> >> implementation so far is not complete and more work is required to
> >> overcome some of its restrictions. While I like your idea I believe that
> >> it imposed too large a
> >> granularity on the encrypted data, all fields will all kinds of data
> >> will be encrypted including  images and others which normally would be
> >> left alone, thus adding to the performance penalty due to encryption.
> >
> > I don't agree with you here. In Lucene, you will encrypt the field data,
> > the
> > field names, and the tokens : I would say that is represents at least 2/3
> > of
> > the index size. Then, with the implementation you suggest, I think (sorry
> > I
> > didn't took time to see you patch) that every time a lucene data need to
> > be
> > read, it is decrypted each time. With an encrypted FS, your kernel will
> > maintain a cache in RAM for you, so it won't hurt so much.
> > It needs some bench to see what is effectively the best, but I have doubt
> > that
> > your solution will be faster.
> >
> > Nicolas.
> Nicolas, I am all in favour of some tests to establish which solution is
> best, but I have to say that I don't believe file system or directory
> encryption in Lucene is really justified. Most operating system already
> provide this feature, although they are system-wide or policy-based
> solution, hence not always within individual user control.
> But if the issue is user control, then I believe Lucene should provide
> maximum granularity when it comes to choice of data to encrypt.
> The issue I believe is whether some form of encryption should be provided
> within Lucene to enable application developers to create applications which
> offer some data protection under user control, with a minimum of impact,
> where by impact I mean both on peformance and workload either in Lucene
> code or user code.

In fact you mean a user that has no control of it's machine, and that cannot 
encrypt his partition. Here you will have the issue with the swap : Lucene 
will decrypt the data in RAM, that can possibly pushed on the swap... I know 
this is extreme, but it's a security hole.

Solutions & Technologies
Tel : +33 (0)5 61 00 52 90
Fax : +33 (0)5 61 00 51 46

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message