jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mat Mannion (JIRA)" <j...@apache.org>
Subject [jira] [Created] (JCLOUDS-1264) Multipart SLO uploads fail if key contains multibyte Unicode characters
Date Mon, 03 Apr 2017 10:43:42 GMT
Mat Mannion created JCLOUDS-1264:
------------------------------------

             Summary: Multipart SLO uploads fail if key contains multibyte Unicode characters
                 Key: JCLOUDS-1264
                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1264
             Project: jclouds
          Issue Type: Bug
          Components: jclouds-blobstore
    Affects Versions: 2.0.1
         Environment: Devstack on Ubuntu 14.04
            Reporter: Mat Mannion


If a large object is PUT to a key that contains multibyte Unicode characters, the manifest
upload fails because the {{Content-Length}} header sent is incorrect.

A small project that reproduces the issue against the latest Devstack Swift can be found here:
https://github.com/UniversityofWarwick/jclouds-highbyte-slo-bug

In short:

{noformat}
>> "[{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000001","etag":"58f06dd588d8ffb3beb46ada6309436b","size_bytes":33554432},{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000002","etag":"2b4b81733d0a2e4abe89516639627408","size_bytes":4993481}]"
>> PUT http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put
HTTP/1.1
>> Content-Length: 272

14657 ERROR org.jclouds.http.internal.JavaUrlHttpCommandExecutorService error after writing
0/272 bytes to http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put
java.io.IOException: too many bytes written
	at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3505)
	at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53)
	at com.google.common.io.ByteStreams.copy(ByteStreams.java:179)
	at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:298)
	at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:171)
	at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:65)
	at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:99)
	at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
	at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
	at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
	at com.sun.proxy.$Proxy77.replaceManifest(Unknown Source)
	at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.completeMultipartUpload(RegionScopedSwiftBlobStore.java:522)
	at uk.ac.warwick.slo.AbstractJCloudsSLOTest.putSLO(AbstractJCloudsSLOTest.java:70)
	at uk.ac.warwick.slo.AbstractJCloudsSLOTest.assertCanPutAndGetSLO(AbstractJCloudsSLOTest.java:74)
	at uk.ac.warwick.slo.AbstractJCloudsSLOTest.sloWithHighByteChars(AbstractJCloudsSLOTest.java:91)
	at uk.ac.warwick.slo.SwiftJCloudsSLOTest.sloWithHighByteChars(SwiftJCloudsSLOTest.java:7)
{noformat}

The cause of this is {{org.jclouds.openstack.swift.v1.binders.BindManifestToJsonPayload:61}},
where the ContentLength is set to the length of the JSON String in Java, which is not the
same as the String byte length. This is actually unnecessary, as {{request.setPayload(String)}}
creates a {{StringPayload}}, and the constructor here sets the content length to the correct
byte length - so I think this code is left-over from a previous version where this isn't the
case.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message