jclouds-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Archana C <carchan...@yahoo.co.uk>
Subject jClouds 2.0 MultiPart Upload
Date Fri, 03 Feb 2017 05:22:15 GMT
Hi 

I have written a sample code for multipart upload using jClouds-2.0
        Properties overrides = new Properties();
        BlobStoreContext context = ContextBuilder.newBuilder("openstack-swift")
                .endpoint("http://x.xxx.xx.xx:5000/v2.0")
                .credentials("xx:xx", "xx")
                .overrides(overrides)
                .modules(modules)
                .buildView(BlobStoreContext.class);
        BlobStore blobStore = context.getBlobStore();
        blobStore.createContainerInLocation(null, CONTAINER_NAME);
        Path path = Paths.get("test2");
        File f = new File("test2");
        byte []byteArray =  Files.readAllBytes(path);
        Payload payload = newByteSourcePayload(wrap(byteArray));
        PutOptions opt = new PutOptions();
        opt.multipart();
        Blob blob = blobStore.blobBuilder(OBJECT_NAME)
                .payload(payload).contentLength(f.length())
                .build();
        String etag =  blobStore.putBlob(CONTAINER_NAME, blob, opt);
test2 is the file I am trying to upload which is of size 36MB and I am getting the following
exception
10:21:52.355 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request 1344471693:
PUT http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP HTTP/1.1
10:21:52.356 [main] DEBUG jclouds.headers - >> PUT http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP
HTTP/1.1
10:21:52.356 [main] DEBUG jclouds.headers - >> Accept: application/json
10:21:52.357 [main] DEBUG jclouds.headers - >> X-Auth-Token: fd72b74db90c46cabcca3f317d5a09d4
10:21:53.129 [main] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Receiving response 1344471693:
HTTP/1.1 201 Created
10:21:53.129 [main] DEBUG jclouds.headers - << HTTP/1.1 201 Created
10:21:53.129 [main] DEBUG jclouds.headers - << Date: Fri, 03 Feb 2017 04:51:53 GMT
10:21:53.129 [main] DEBUG jclouds.headers - << X-Trans-Id: tx83ba6249347c43c99bb41-0058940c68
10:21:53.129 [main] DEBUG jclouds.headers - << Connection: keep-alive
10:21:53.129 [main] DEBUG jclouds.headers - << Content-Type: text/html; charset=UTF-8
10:21:53.129 [main] DEBUG jclouds.headers - << Content-Length: 0        ------->
Container Creation Successful
10:21:53.373 [user thread 1] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking
object:put
10:21:53.373 [user thread 0] DEBUG o.j.rest.internal.InvokeHttpMethod - >> invoking
object:put
10:21:53.374 [user thread 1] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request
823625484: PUT http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP/arc/slo/1486097513.327000/0/33554432/00000001
HTTP/1.1
10:21:53.376 [user thread 0] DEBUG o.j.h.i.JavaUrlHttpCommandExecutorService - Sending request
-1220101806: PUT http://x.x.x.x:8091/v1/AUTH_0909ac10e7024847b1a9fe9787c7de8f/arctestMP/arc/slo/1486097513.327000/0/33554432/00000000
HTTP/1.1
10:21:53.396 [user thread 1] DEBUG org.jclouds.http.internal.HttpWire - over limit 3145728/262144:
wrote temp file
10:21:53.552 [user thread 0] DEBUG org.jclouds.http.internal.HttpWire - over limit 33554432/262144:
wrote temp fileException in thread "main" com.google.common.util.concurrent.ExecutionError:
java.lang.OutOfMemoryError: Java heap space
    at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1380)
    at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1373)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:650)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:628)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putBlob(RegionScopedSwiftBlobStore.java:274)
    at jclouds20.App.main(App.java:83)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.lang.StringBuilder.ensureCapacityImpl(StringBuilder.java:342)
    at java.lang.StringBuilder.append(StringBuilder.java:208)
    at org.jclouds.logging.internal.Wire.wire(Wire.java:68)
    at org.jclouds.logging.internal.Wire.copy(Wire.java:99)
    at org.jclouds.logging.internal.Wire.output(Wire.java:176)
    at org.jclouds.logging.internal.Wire.output(Wire.java:143)
    at org.jclouds.http.HttpUtils.wirePayloadIfEnabled(HttpUtils.java:296)
    at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:97)
    at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
    at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
    at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
    at com.sun.proxy.$Proxy68.put(Unknown Source)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.uploadMultipartPart(RegionScopedSwiftBlobStore.java:529)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:666)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:653)
    at java.util.concurrent.FutureTask.run(FutureTask.java:274)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
    at java.lang.Thread.run(Thread.java:798)
    at java.lang.Thread.getStackTraceImpl(Native Method)
    at java.lang.Thread.getStackTrace(Thread.java:1133)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:647)
    ... 3 more
Feb 03, 2017 10:22:03 AM com.google.common.util.concurrent.Futures$CombinedFuture setExceptionAndMaybeLog
SEVERE: input future failed.
java.lang.OutOfMemoryError: Java heap space
    at java.lang.StringBuilder.ensureCapacityImpl(StringBuilder.java:342)
    at java.lang.StringBuilder.append(StringBuilder.java:208)
    at org.jclouds.logging.internal.Wire.wire(Wire.java:68)
    at org.jclouds.logging.internal.Wire.copy(Wire.java:99)
    at org.jclouds.logging.internal.Wire.output(Wire.java:176)
    at org.jclouds.logging.internal.Wire.output(Wire.java:143)
    at org.jclouds.http.HttpUtils.wirePayloadIfEnabled(HttpUtils.java:296)
    at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:97)
    at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
    at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
    at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
    at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
    at com.sun.proxy.$Proxy68.put(Unknown Source)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.uploadMultipartPart(RegionScopedSwiftBlobStore.java:529)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:666)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore$BlobUploader.call(RegionScopedSwiftBlobStore.java:653)
    at java.util.concurrent.FutureTask.run(FutureTask.java:274)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627)
    at java.lang.Thread.run(Thread.java:798)
    at java.lang.Thread.getStackTraceImpl(Native Method)
    at java.lang.Thread.getStackTrace(Thread.java:1133)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:647)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putMultipartBlob(RegionScopedSwiftBlobStore.java:628)
    at org.jclouds.openstack.swift.v1.blobstore.RegionScopedSwiftBlobStore.putBlob(RegionScopedSwiftBlobStore.java:274)
    at jclouds20.App.main(App.java:83)

Should putBlob with multipart enabled in PutOptions not be used for uploading files greater
than 32MB from jClouds-2.0 ?
Is it mandatory to use Swift API's initiate, upload and complete multipart from jClouds-2.0
?

RegardsArchana



Mime
View raw message