drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Drill to query Client-side encrypted data from S3
Date Tue, 07 Apr 2015 22:29:53 GMT
Looking at the link that you provided, it appears that you are encrypting
entire data files.  That probably makes it better to implement this as a
layer in the file access path.

Drill doesn't do this just now, but it would be relatively easy to add, I
think.



On Tue, Apr 7, 2015 at 3:26 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

>
> Ahh...
>
> There is no magic that will handle decryption that you can plug into (at
> this time).
>
>
>
> On Tue, Apr 7, 2015 at 3:02 PM, Ganesha Muthuraman <mganesh123@outlook.com
> > wrote:
>
>> The situation is this:
>> There is client side encrypted data on S3. There is an EMR cluster that
>> uses this as EMRFS. The EMR client reaches out to a custom java class for
>> decrypting it. EMR does it using the envelope encryption method, documented
>> on AWS.
>> http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-plan-cse.html
>> My question was, is there a way that I can use the custom java module
>> that I have (aka EncryptionProvider) to work with Drill so that I can
>> achieve the same kind of envelope decryption that EMR does? Or does it have
>> to be a completely new UDF that I use that in turn calls a custom Java
>> module that can decrypt this data? Apologies if my message is confusing.
>> -Ganesh
>> > Subject: Re: Drill to query Client-side encrypted data from S3
>> > From: dtucker@maprtech.com
>> > Date: Tue, 7 Apr 2015 14:47:39 -0700
>> > To: user@drill.apache.org
>> >
>> > Ganesh,
>> >
>> > When you say the keys are “custom controlled”, does that mean that only
>> special logic within your Java application allows the data to be properly
>> accessed ?   There are several mechanisms within the S3 API such that
>> encryption/decryption occur transparently to the application.   If your
>> data is accessible in that manner, it’s likely that simply setting the
>> correct properties and jar files for your Drill environment will allow your
>> queries to access the data.
>> >
>> > — David
>> >
>> > On Apr 7, 2015, at 2:41 PM, Ganesha Muthuraman <mganesh123@outlook.com>
>> wrote:
>> >
>> > > I am trying to use Drill to read from Amazon S3 where the data is
>> Client-side encrypted, meaning the keys to decrypt the data are custom
>> controlled. Is there a way I can use drill with this data given that I have
>> a java module that can be called that will provide the master key to
>> decrypt the data on the fly?
>> > > My situation: A lot of the use cases that we have might work well
>> with the new approach of S3 client-side encryption, but for using drill to
>> explore that data. So any pointers/help here will be much appreciated.
>> > > Thanks!
>> > > -Ganesh
>> >
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message