hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16276) Fix NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
Date Wed, 22 Mar 2017 21:10:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15937137#comment-15937137
] 

Edward Capriolo commented on HIVE-16276:
----------------------------------------

I mean to say. I already put s3 jars in my classpath and set the location. CREATE TABLE edstuff
(x) LOCATION 's3://whereever' along with setting the keys. My understanding of the process
is the drivers register themselves, by saying for s3:// use me. Wont we have the situation
now where two jars are looking to register themselves as s3 drivers?

> Fix NoSuchMethodError: com.amazonaws.services.s3.transfer.TransferManagerConfiguration.setMultipartUploadThreshold(I)V
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-16276
>                 URL: https://issues.apache.org/jira/browse/HIVE-16276
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>         Attachments: HIVE-16276.1.patch, HIVE-16276.2.patch, HIVE-16276.3.patch, HIVE-16276.4.patch
>
>
> The druid-handler is pulling in some com.amazonaws dependencies that conflict with the
version that Hadoop is using. This causes the above exception to be thrown when running Hive
against S3. This patch fixes the dependency issue by shading the aws dependencies in the druid
artifacts. Unfortunately, I can't find a great way to add a test for this so it doesn't happen
in the future. We will need some more robust S3-integration tests for that.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message