jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timur Alperovich <notificati...@github.com>
Subject Re: [jclouds] JCLOUDS-894: Add portable multipart upload (#762)
Date Wed, 03 Jun 2015 22:35:53 GMT
AWS Java S3 SDK does the following:

    public static long calculateOptimalPartSize(PutObjectRequest putObjectRequest, TransferManagerConfiguration
configuration) {
        double contentLength = TransferManagerUtils.getContentLength(putObjectRequest);
        double optimalPartSize = (double)contentLength / (double)MAXIMUM_UPLOAD_PARTS;
         // round up so we don't push the upload over the maximum number of parts
        optimalPartSize = Math.ceil(optimalPartSize);
        return (long)Math.max(optimalPartSize, configuration.getMinimumUploadPartSize());

AWS SDK defaults to a maximum of 10000 parts. The minimum default part size is 5MB. So, uploading
a 51GB file, for example, would use 8500 6MB parts.

Jclouds could use a similar mechanism. It would probably make sense to expose the configuration
parameters to be able to change the default behavior.

Reply to this email directly or view it on GitHub:
View raw message