libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LIBCLOUD-834) AWS Storage S3 multipart uploads fail when using v4 authentication
Date Thu, 09 Mar 2017 05:27:38 GMT

    [ https://issues.apache.org/jira/browse/LIBCLOUD-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902520#comment-15902520
] 

ASF GitHub Bot commented on LIBCLOUD-834:
-----------------------------------------

GitHub user amisstea opened a pull request:

    https://github.com/apache/libcloud/pull/1005

    [LIBCLOUD-834] Reintroduce S3 multipart upload support with signature v4

    ## Reintroduce S3 multipart upload support with signature v4
    
    ### Description
    
    All S3 regions now support signature V4. Rather than try to implement multipart
    uploads with both V2 and V4, all region drivers have been converted to use
    the latter. All multipart changes are based on code which existed in v1.5.0.
    
    Some minor adjustments were made to the use of requests made throughout
    the BaseS3StorageDriver class. The practice of constructing request paths with
    parameters included needed to be removed. It causes issues during the
    signature calculation. The solution is to use the params argument instead.
    In addition, a fix was necessary in the signature calculation to not compute a
    payload hash in the event of file uploads. It seems this was already taken into
    consideration but failed to account for the scenario where an iterator is
    passed to the data argument.
    
    I have tested these changes against S3 using each of the following methods and with each
regional driver.
    
    get_container
    create_container
    delete_container
    list_container_objects
    get_object
    delete_object
    upload_object
    upload_object_via_stream
    download_object
    download_object_via_stream
    ex_iterate_multipart_uploads
    ex_cleanup_all_multipart_uploads
    
    ### Status
    
    - done, ready for review
    
    ### Checklist (tick everything that applies)
    
    - [x] [Code linting](http://libcloud.readthedocs.org/en/latest/development.html#code-style-guide)
(required, can be done after the PR checks)
    - [x] Documentation
    - [x] [Tests](http://libcloud.readthedocs.org/en/latest/testing.html)
    - [ ] [ICLA](http://libcloud.readthedocs.org/en/latest/development.html#contributing-bigger-changes)
(required for bigger changes)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/amisstea/libcloud libcloud-834-s3-multipart-upload-v4

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/libcloud/pull/1005.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1005
    
----
commit 01967258b3ae5f4307ca352180d1098c10209bbb
Author: Alex Misstear <amisstea@redhat.com>
Date:   2017-03-08T21:29:27Z

    [LIBCLOUD-834] Reintroduce S3 multipart upload support with signature v4
    
    All S3 regions now support signature V4. Rather than try to implement multipart
    uploads with both V2 and V4, all region drivers have been converted to use
    the latter. All multipart changes are based on code which existed in v1.5.0.
    
    Some minor adjustments were made to the use of requests made throughout
    the BaseS3StorageDriver class. The practice of constructing request paths with
    parameters included needed to be removed. It causes issues during the
    signature calculation. The solution is to use the params argument instead.
    In addition, a fix was necessary in the signature calculation to not compute a
    payload hash in the event of file uploads. It seems this was already taken into
    consideration but failed to account for the scenario where an iterator is
    passed to the data argument.

----


> AWS Storage S3 multipart uploads fail when using v4 authentication 
> -------------------------------------------------------------------
>
>                 Key: LIBCLOUD-834
>                 URL: https://issues.apache.org/jira/browse/LIBCLOUD-834
>             Project: Libcloud
>          Issue Type: Bug
>          Components: Storage
>         Environment: Production
>            Reporter: Luke Morfitt
>            Priority: Blocker
>
> When using v4 authentication and attempting to perform a object upload using "upload_object_via_stream"
the upload fails with the following error.
> I suspect the issue relates to the PUT or POST method with v4 authentication, however
I have not tracked it down.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message