libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [libcloud] dvdrbn-m9 opened a new issue #1562: Problem uploading empty files with upload_object
Date Thu, 04 Mar 2021 10:46:13 GMT

dvdrbn-m9 opened a new issue #1562:
URL: https://github.com/apache/libcloud/issues/1562


   ## Summary
   
   Using `upload_object` to upload an empty file returns response:
   `400 - Bad Request: too many Host headers`
   
   ## Detailed Information
   
   Python 3.6.8
   Libcloud: 3.3.1
   OS: rhel 7.9
   
   I use libcloud to upload objects to 3 providers - aws s3, minio & netapp storageGrid,
I use the s3 driver for all of them. `upload_object` function works fine except when I try
to upload an empty file (I sometimes need to store just meta-data with a specific object name
for later use). When trying to upload an empty file I get the response:
   ```
   status: 400
   reason: 'Bad Request: too many Host headers'
   ```
   
   If I add anything to the file's content it uploads successfully with the meta-data, so
its not a problem with permissions/path in s3 or something like that. I also tried without
the meta-data, but that didn't work either. 
   I tried using `upload_object_via_stream` and found the following code works:
   ```
   with io.BytesIO() as iterator:
       driver.upload_object_via_stream(iterator=iterator,
                                       container=container,
                                       object_name=objName,
                                       extra=extra)
   ```
   This code uploads a 0 bytes file successfully to aws s3 & minio, but not to netapp's
storageGrid (so unfortunately I can't use it).
   I think it's a limitation of storageGrid's multi-part upload and not some other storage
restriction (we have existing empty objects on storageGrid, but those were uploaded using
jclouds).
   
   I think using `upload_object` will be the best solution for all 3 providers, but I don't
understand why it's not uploading empty files and would appreciate help with this. Thanks!
   
   ### Reproduce
   1. `touch ~/test-empty.txt`
   2. Use `upload_object` to upload test-empty.txt
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message