jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From dordoriko <notificati...@github.com>
Subject [jclouds/jclouds] JCLOUDS-847: Poor upload performance for putBlob (#1274)
Date Tue, 23 Apr 2019 15:37:52 GMT
This change improves the performance of writing to sockets with the
default Java URL connection HTTP client, by enlarging the buffer used
for socket writes from an implicit hard-coded 4KB / 8KB buffer to a
configurable 32KB buffer.

The buffer size is now controlled by the following property with the
following default value:

jclouds.output-socket-buffer-size: 32768

The implementation is based on a variant of ByteStreams.copy (written as
ByteStreams2.copy) which accepts the buffer size as an argument, unlike
the original Guava code that uses a hard-coded size.

The change was done directly within the loop that copies the input
stream to the output stream, and not by wrapping a BufferedOutputStream
around the existing output stream, in order to avoid copying the payload
twice.

On some platforms this change can improve both the putBlob throughput
and the total CPU consumption.
You can view, comment on, or merge this pull request online at:

  https://github.com/jclouds/jclouds/pull/1274

-- Commit Summary --

  * JCLOUDS-847: Poor upload performance for putBlob

-- File Changes --

    M core/src/main/java/org/jclouds/Constants.java (9)
    M core/src/main/java/org/jclouds/apis/internal/BaseApiMetadata.java (2)
    M core/src/main/java/org/jclouds/http/internal/JavaUrlHttpCommandExecutorService.java
(8)
    M core/src/main/java/org/jclouds/io/ByteStreams2.java (23)
    M core/src/test/java/org/jclouds/http/internal/TrackingJavaUrlHttpCommandExecutorService.java
(4)
    M providers/dynect/src/main/java/org/jclouds/dynect/v3/config/DynECTHttpApiModule.java
(4)
    M providers/profitbricks/src/main/java/org/jclouds/profitbricks/http/ResponseStatusFromPayloadHttpCommandExecutorService.java
(4)

-- Patch Links --

https://github.com/jclouds/jclouds/pull/1274.patch
https://github.com/jclouds/jclouds/pull/1274.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1274

Mime
View raw message