hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph Lawson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-3733) "s3:" URLs break when Secret Key contains a slash, even if encoded
Date Thu, 01 Aug 2013 19:05:52 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13726760#comment-13726760
] 

Joseph Lawson commented on HADOOP-3733:
---------------------------------------

I would like to comment that this bug will bite anyone using AWS IAM credentials more often
that one may think.  Considering that there are 40 characters in the IAM private key and 64
characters in the total choices, there is a 62.5% chance that a / is going to appear in the
private key.  So basically there is a 62% chance that hadoop will fail on AWS for any person
using this method of access.  Seems a bit more than a low priority bug.
                
> "s3:" URLs break when Secret Key contains a slash, even if encoded
> ------------------------------------------------------------------
>
>                 Key: HADOOP-3733
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3733
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 0.17.1, 2.0.2-alpha
>            Reporter: Stuart Sierra
>            Priority: Minor
>         Attachments: HADOOP-3733-20130223T011025Z.patch, hadoop-3733.patch, HADOOP-3733.patch
>
>
> When using URLs of the form s3://ID:SECRET@BUCKET/ at the command line, distcp fails
if the SECRET contains a slash, even when the slash is URL-encoded as %2F.
> Say your AWS Access Key ID is RYWX12N9WCY42XVOL8WH
> And your AWS Secret Key is Xqj1/NMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv
> And your bucket is called "mybucket"
> You can URL-encode the Secret KKey as Xqj1%2FNMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv
> But this doesn't work:
> {noformat}
> $ bin/hadoop distcp file:///source  s3://RYWX12N9WCY42XVOL8WH:Xqj1%2FNMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv@mybucket/dest
> 08/07/09 15:05:22 INFO util.CopyFiles: srcPaths=[file:///source]
> 08/07/09 15:05:22 INFO util.CopyFiles: destPath=s3://RYWX12N9WCY42XVOL8WH:Xqj1%2FNMvKBhl1jqKlzbYJS66ua0e8z7Kkvptl9bv@mybucket/dest
> 08/07/09 15:05:23 WARN httpclient.RestS3Service: Unable to access bucket: mybucket
> org.jets3t.service.S3ServiceException: S3 HEAD request failed. ResponseCode=403, ResponseMessage=Forbidden
>         at org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:339)
> ...
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: org.apache.hadoop.fs.s3.S3Exception: org.jets3t.service.S3ServiceException:
S3 PUT failed. XML Error Message: <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch</Code><Message>The
request signature we calculated does not match the signature you provided. Check your key
and signing method.</Message>
>         at org.apache.hadoop.fs.s3.Jets3tFileSystemStore.createBucket(Jets3tFileSystemStore.java:141)
> ...
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message