jclouds-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Gaul (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (JCLOUDS-1264) Multipart SLO uploads fail if key contains multibyte Unicode characters
Date Tue, 04 Apr 2017 12:34:41 GMT

     [ https://issues.apache.org/jira/browse/JCLOUDS-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Gaul resolved JCLOUDS-1264.
----------------------------------
       Resolution: Fixed
         Assignee: Andrew Gaul
    Fix Version/s: 2.1.0

> Multipart SLO uploads fail if key contains multibyte Unicode characters
> -----------------------------------------------------------------------
>
>                 Key: JCLOUDS-1264
>                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1264
>             Project: jclouds
>          Issue Type: Bug
>          Components: jclouds-blobstore
>    Affects Versions: 2.0.1
>         Environment: Devstack on Ubuntu 14.04
>            Reporter: Mat Mannion
>            Assignee: Andrew Gaul
>             Fix For: 2.1.0
>
>
> If a large object is PUT to a key that contains multibyte Unicode characters, the manifest
upload fails because the {{Content-Length}} header sent is incorrect.
> A small project that reproduces the issue against the latest Devstack Swift can be found
here: https://github.com/UniversityofWarwick/jclouds-highbyte-slo-bug
> In short:
> {noformat}
> >> "[{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000001","etag":"58f06dd588d8ffb3beb46ada6309436b","size_bytes":33554432},{"path":"slo-test/Ni[0xc5][0x9f]an.rar/slo/1491214092.299000/38547913/0/00000002","etag":"2b4b81733d0a2e4abe89516639627408","size_bytes":4993481}]"
> >> PUT http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put
HTTP/1.1
> >> Content-Length: 272
> 14657 ERROR org.jclouds.http.internal.JavaUrlHttpCommandExecutorService error after writing
0/272 bytes to http://137.205.194.8:8080/v1/AUTH_c7a3e66567ec442080a360d6d23f2dbe/slo-test/Ni%C5%9Fan.rar?multipart-manifest=put
> java.io.IOException: too many bytes written
> 	at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3505)
> 	at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53)
> 	at com.google.common.io.ByteStreams.copy(ByteStreams.java:179)
> 	at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:298)
> 	at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:171)
> 	at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:65)
> 	at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:99)
> 	at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
> 	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
> 	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
> 	at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
> 	at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
> 	at com.sun.proxy.$Proxy77.replaceManifest(Unknown Source)
> 	at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.completeMultipartUpload(RegionScopedSwiftBlobStore.java:522)
> 	at uk.ac.warwick.slo.AbstractJCloudsSLOTest.putSLO(AbstractJCloudsSLOTest.java:70)
> 	at uk.ac.warwick.slo.AbstractJCloudsSLOTest.assertCanPutAndGetSLO(AbstractJCloudsSLOTest.java:74)
> 	at uk.ac.warwick.slo.AbstractJCloudsSLOTest.sloWithHighByteChars(AbstractJCloudsSLOTest.java:91)
> 	at uk.ac.warwick.slo.SwiftJCloudsSLOTest.sloWithHighByteChars(SwiftJCloudsSLOTest.java:7)
> {noformat}
> The cause of this is {{org.jclouds.openstack.swift.v1.binders.BindManifestToJsonPayload:61}},
where the ContentLength is set to the length of the JSON String in Java, which is not the
same as the String byte length. This is actually unnecessary, as {{request.setPayload(String)}}
creates a {{StringPayload}}, and the constructor here sets the content length to the correct
byte length - so I think this code is left-over from a previous version where this isn't the
case.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message