lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kumaran Ramasubramanian <kums....@gmail.com>
Subject Re: Encryption at lucene index
Date Fri, 11 Aug 2017 18:42:44 GMT
I got it Erick.. Thank you..

-
‚ÄčKumaran R
‚Äč

On Fri, Aug 11, 2017 at 10:35 PM, Erick Erickson <erickerickson@gmail.com>
wrote:

> Encrypting the _tokens_ inevitably leads to reduced capabilities BTW.
> Trivial example:
> I have these tokens in my index
> run
> runner
> running
> runs
>
> Any non-trivial encryption algorithm will not encrypt the first three
> letters "run" identically in all three so searching for run* simply
> won't work.
>
> As you can see, there's quite a bit of back-and-forth with that JIRA
> and it is pretty much been abandoned.
>
> Best,
> Erick
>
> On Thu, Aug 10, 2017 at 11:17 PM, Kumaran Ramasubramanian
> <kums.134@gmail.com> wrote:
> > Hi Ishan, thank you :-)
> >
> > -
> > -
> > Kumaran R
> >
> >
> >
> > On Mon, Aug 7, 2017 at 10:53 PM, Ishan Chattopadhyaya <
> > ichattopadhyaya@gmail.com> wrote:
> >
> >> Harry Ochiai (Hitachi) has some index encryption solution,
> >> https://www.slideshare.net/maggon/securing-solr-search-
> data-in-the-cloud
> >> I think it is proprietary, but I'm not sure. Maybe more googling might
> help
> >> find the exact page where his solution is described.
> >>
> >> On Mon, Aug 7, 2017 at 9:59 PM, Kumaran Ramasubramanian <
> >> kums.134@gmail.com>
> >> wrote:
> >>
> >> > Hi Erick, i want to encrypt some fields of an document which has
> personal
> >> > identifiable information ( both indexed and stored data)... for eg:
> >> email,
> >> > mobilenumber etc.. i am able to find LUCENE-6966 alone while googling
> >> it..
> >> > any related pointers in solr or latest lucene version?
> >> >
> >> >
> >> > -
> >> > -
> >> > Kumaran R
> >> >
> >> > On Mon, Aug 7, 2017 at 9:52 PM, Erick Erickson <
> erickerickson@gmail.com>
> >> > wrote:
> >> >
> >> > > No, since you haven't defined what you want to encrypt, what your
> >> > > requirements are, what you hope to get out of "encryption" etc.
> >> > >
> >> > > Put the index on an encrypting filesystem and forget about it if you
> >> > > possibly can, because anything else is a significant amount of work.
> >> > > To encrypt the searchable tokens on a per-user basis in memory is
a
> >> > > _lot_ of work. It depends on your security needs.
> >> > >
> >> > > Otherwise, as I said, please ask specific questions as the topic is
> >> > > quite large, much too large to conduct a seminar through the user's
> >> > > list.
> >> > >
> >> > > Best,
> >> > > Erick
> >> > >
> >> > > On Mon, Aug 7, 2017 at 9:07 AM, Kumaran Ramasubramanian
> >> > > <kums.134@gmail.com> wrote:
> >> > > > Hi Erick,
> >> > > >
> >> > > >     Thanks for the information. Any pointers about encryption
> options
> >> > in
> >> > > > solr?
> >> > > >
> >> > > >
> >> > > > --
> >> > > > Kumaran R
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Aug 7, 2017 at 9:17 PM, Erick Erickson <
> >> > erickerickson@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > >> Encryption in Solr has a bunch of ramifications. Do you care
> about
> >> > > >>
> >> > > >> - encryption at rest or in memory?
> >> > > >> - encrypting the _searchable_ tokens?
> >> > > >> - encrypting the searchable tokens per-user?
> >> > > >> - encrypting the stored data (which a filter won't do BTW).
> >> > > >>
> >> > > >> It's actually a fairly complex topic the discussion at
> LUCENE-6966
> >> > > >> outlines much of it. Please ask specific questions as you
> research
> >> the
> >> > > >> topic. One  per-user encryption package that I know of is
by
> Hitachi
> >> > > >> Solutions (commercial) and it explicitly does _not_ support,
for
> >> > > >> instance, wildcards (there are other limitations too). See:
> >> > > >> http://www.hitachi-solutions.com/securesearch/
> >> > > >>
> >> > > >> Most of the time when people ask for encryption they soon
> discover
> >> > > >> it's much more difficult than they imagine and settle for
just
> >> putting
> >> > > >> the indexes on an encrypting file system. When they move
beyond
> that
> >> > > >> it gets complex and you'd be well advised to consult with
Solr
> >> > > >> security experts.
> >> > > >>
> >> > > >> Best,
> >> > > >> Erick
> >> > > >>
> >> > > >> On Sun, Aug 6, 2017 at 11:30 PM, Kumaran Ramasubramanian
> >> > > >> <kums.134@gmail.com> wrote:
> >> > > >> > Hi All,
> >> > > >> >
> >> > > >> >
> >> > > >> > After looking at all below discussions, i have one doubt
which
> may
> >> > be
> >> > > >> silly
> >> > > >> > or novice but i want to throw this to lucene user list.
> >> > > >> >
> >> > > >> > if we have encryption layer included in our analyzer's
flow of
> >> > filters
> >> > > >> like
> >> > > >> > EncryptionFilter to control field-level encryption.
what are
> the
> >> > > >> > consequences ? am i missing anything basic?
> >> > > >> >
> >> > > >> > Thanks in advance..
> >> > > >> >
> >> > > >> >
> >> > > >> > Related links:
> >> > > >> >
> >> > > >> > https://issues.apache.org/jira/browse/LUCENE-2228 :
AES
> Encrypted
> >> > > >> Directory
> >> > > >> > - in lucene 3.x
> >> > > >> >
> >> > > >> > https://issues.apache.org/jira/browse/LUCENE-6966 :
 Codec for
> >> > > >> index-level
> >> > > >> > encryption - at codec level, to have control on which
column /
> >> field
> >> > > have
> >> > > >> >  personal identifiable information
> >> > > >> >
> >> > > >> > https://security.stackexchange.com/questions/
> >> > > 111153/is-a-lucene-search-
> >> > > >> index-effectively-a-backdoor-for-field-level-encryption
> >> > > >> >
> >> > > >> >
> >> > > >> > A decent encrypting algorithm will not produce, say,
the same
> >> first
> >> > > >> portion
> >> > > >> >> for two tokens that start with the same letters.
So wildcard
> >> > searches
> >> > > >> won't
> >> > > >> >> work. Consider "runs", "running", "runner". A search
on "run*"
> >> > would
> >> > > be
> >> > > >> >> expected to match all three, but wouldn't unless
the
> encryption
> >> > were
> >> > > so
> >> > > >> >> trivial as to be useless. Similar issues arise with
sorting.
> >> "More
> >> > > Like
> >> > > >> >> This" would be unreliable. There are many other
features of a
> >> > robust
> >> > > >> search
> >> > > >> >> engine that would be impacted, and an index with
encrypted
> terms
> >> > > would
> >> > > >> be
> >> > > >> >> useful for only exact matches, which usually results
in a poor
> >> > search
> >> > > >> >> experience.
> >> > > >> >
> >> > > >> >
> >> > > >> > https://stackoverflow.com/questions/36604551/adding-
> >> > > >> encryption-to-solr-lucene-indexes
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > --
> >> > > >> > Kumaran R
> >> > > >>
> >> > > >> ------------------------------------------------------------
> >> ---------
> >> > > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > > >> For additional commands, e-mail: java-user-help@lucene.apache.
> org
> >> > > >>
> >> > > >>
> >> > >
> >> > > ------------------------------------------------------------
> ---------
> >> > > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> > > For additional commands, e-mail: java-user-help@lucene.apache.org
> >> > >
> >> > >
> >> >
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message