hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HADOOP-13078) switch hadoop-aws back to using the (heavy) amazon-sdk JAR
Date Thu, 12 May 2016 15:30:13 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran reassigned HADOOP-13078:

    Assignee: Steve Loughran

> switch hadoop-aws back to using the (heavy) amazon-sdk JAR
> ----------------------------------------------------------
>                 Key: HADOOP-13078
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13078
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
> Hadoop 2.6-2.7 uses the full amazon-aws-sdk JAR. Hadoop 2.8+ has switched to the amazon-s3-sdk
jar because it was lighter weight. 
> I want to return to the full JAR before 2.8 switches, for
> # downstream code: if someone is already including/depending-on/upgrading the aws SDK,
switching to the s3 sdk complicates packaging, distribution. If directly depended on via maven
dependencies, it breaks the build
> # some of the 2.8+ patches, e.g. HADOOP-12537, have to add another part of the S3 SDK
to handle temporary credentials. This will make life even more complex downstream
> # if the hadoop-aws module ever adds more stuff (e.g. a s3mper style use of dynamo db
for directory structure storage), then again, more JARs, more complexity.
> Let's just change the build to return to the original JAR. Yes it is heavy, but it will
be a consistent heaviness for all projects downstream.
> This change *must* go in to 2.8 if we don't want to start breaking things

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message