jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Gaul <notificati...@github.com>
Subject Re: [jclouds/jclouds] Filesystem: Fix the MPU ETags to match S3. (#1252)
Date Tue, 30 Oct 2018 19:52:47 GMT
gaul requested changes on this pull request.



>           .append("-")
          .append(partsList.size())
          .append("\"")
          .toString();
       assertThat(blob.getMetadata().getETag()).isEqualTo(expectedETag);
    }
 
+   @Test(groups = { "integration", "live" })
+   public void testMultipartUploadMultiplePartsKnownETag() throws Exception {
+      BlobStore blobStore = view.getBlobStore();
+      String container = getContainerName();
+      // Pre-computed ETag returned by AWS S3 for the MPU consisting of two 5MB parts filled
with 'b'
+      String expectedETag = "\"84462a16f6a60478d50148808aa609c1-2\"";

S3 has emerged as the de facto standard for object storage.  I think the average user would
prefer the local blobstore acts more similarly to S3, especially since one of the main consumers
of this is S3Proxy.  :-)

> @@ -838,10 +839,10 @@ public String completeMultipartUpload(MultipartUpload mpu, List<MultipartPart>
p
             throw propagate(ioe);
          }
          streams.add(is);
-         partHashes.append(blobPart.getMetadata().getETag());
+         md5Hasher.putBytes(new BigInteger(blobPart.getMetadata().getETag(), 16).toByteArray());

This is clever but would `BaseEncoding.base16.decode` address this more naturally?

> +   @Test(groups = { "integration", "live" })
+   public void testMultipartUploadMultiplePartsKnownETag() throws Exception {
+      BlobStore blobStore = view.getBlobStore();
+      String container = getContainerName();
+      // Pre-computed ETag returned by AWS S3 for the MPU consisting of two 5MB parts filled
with 'b'
+      String expectedETag = "\"84462a16f6a60478d50148808aa609c1-2\"";
+      int partSize = 5 * (int) Math.pow(2, 20);
+      try {
+         String name = "blob-name";
+         BlobBuilder blobBuilder = blobStore.blobBuilder(name);
+         Blob blob = blobBuilder.build();
+         MultipartUpload mpu = blobStore.initiateMultipartUpload(container, blob.getMetadata(),
new PutOptions());
+
+         byte[] content = new byte[partSize];
+         Arrays.fill(content, (byte) 'b');
+         Payload payload = Payloads.newByteArrayPayload(content);

Would `TestUtils.randomdByteSource(seed)` work as well?

>           .append("-")
          .append(partsList.size())
          .append("\"")
          .toString();
       assertThat(blob.getMetadata().getETag()).isEqualTo(expectedETag);
    }
 
+   @Test(groups = { "integration", "live" })
+   public void testMultipartUploadMultiplePartsKnownETag() throws Exception {
+      BlobStore blobStore = view.getBlobStore();
+      String container = getContainerName();
+      // Pre-computed ETag returned by AWS S3 for the MPU consisting of two 5MB parts filled
with 'b'
+      String expectedETag = "\"84462a16f6a60478d50148808aa609c1-2\"";
+      int partSize = 5 * (int) Math.pow(2, 20);

Easier to read `5 * 1024 * 1024`.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1252#pullrequestreview-169960399
Mime
View raw message