libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [libcloud] Kami commented on a change in pull request #1287: [LIBCLOUD-1043] Fix Azure upload_object_via_stream used with iter
Date Sun, 26 May 2019 19:36:12 GMT
Kami commented on a change in pull request #1287: [LIBCLOUD-1043] Fix Azure upload_object_via_stream
used with iter
URL: https://github.com/apache/libcloud/pull/1287#discussion_r287611382
 
 

 ##########
 File path: libcloud/storage/drivers/azure_blobs.py
 ##########
 @@ -825,7 +826,12 @@ def upload_object_via_stream(self, iterator, container, object_name,
         """
         self._check_values(ex_blob_type, ex_page_blob_size)
         if ex_blob_type == "BlockBlob":
-            iterator.seek(0, os.SEEK_END)
+            try:
+                iterator.seek(0, os.SEEK_END)
+            except AttributeError:
+                buffer = BytesIO()
+                buffer.writelines(iterator)
 
 Review comment:
   I'm fine with this for now, but at the very least, we should add a note to the documentation
(and docstring) which points out that using that method will buffer the whole object in memory.
   
   I do think that using put block would be a good future improvement (especially for very
large objects). IIRC, one of the existing storage drivers already utilizes tsimilar approach,
but I can't recall which one (it does require a bit more work though to track all the chunks,
etc.).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message