hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Moist (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13887) Encrypt S3A data client-side with AWS SDK
Date Wed, 01 Nov 2017 16:52:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234365#comment-16234365

Steve Moist commented on HADOOP-13887:

>I can see the appeal of some form of support for this purely for some backup/restore process,
I agree, that's a scenario I am going to cover in the other proposal.

>People will end up encrypting their data, then be filing bugs/support calls trying to
understand why their queries are all failing.
Oh yes they will.

>It also isn't going to interact with any other S3 client, which is a significant limitation
The aws S3 cse sdk also has that limitation.  IIRC it is also written in Java which makes
portability a concern.  At least with the Hadoop KMS, it exposes REST endpoints to encrypt/decrypt
keys making it more platform independent.  So while utitlities don't integrate currently with
it, it doesn't prevent them from in the future from doing so.  Even a lot of the AWS services
don't integrate with the cse sdk.  

I created HADOOP-15006 and renamed this jira.  I will let [~Igor Mazur] or [~steve_l] close
the ticket as I am unsure of how to do so.

> Encrypt S3A data client-side with AWS SDK
> -----------------------------------------
>                 Key: HADOOP-13887
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13887
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Jeeyoung Kim
>            Assignee: Igor Mazur
>            Priority: Minor
>         Attachments: HADOOP-13887-002.patch, HADOOP-13887-007.patch, HADOOP-13887-branch-2-003.patch,
HADOOP-13897-branch-2-004.patch, HADOOP-13897-branch-2-005.patch, HADOOP-13897-branch-2-006.patch,
HADOOP-13897-branch-2-008.patch, HADOOP-13897-branch-2-009.patch, HADOOP-13897-branch-2-010.patch,
HADOOP-13897-branch-2-012.patch, HADOOP-13897-branch-2-014.patch, HADOOP-13897-trunk-011.patch,
HADOOP-13897-trunk-013.patch, HADOOP-14171-001.patch, S3-CSE Proposal.pdf
> Expose the client-side encryption option documented in Amazon S3 documentation  - http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html
> Currently this is not exposed in Hadoop but it is exposed as an option in AWS Java SDK,
which Hadoop currently includes. It should be trivial to propagate this as a parameter passed
to the S3client used in S3AFileSystem.java

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message