jclouds-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Hsueh <daniel.hs...@seagate.com>
Subject java.net.SocketTimeoutException correlated with old generation GC
Date Mon, 17 Nov 2014 19:49:05 GMT

I'm experiencing behavior I can't explain and I'm wondering if you have any
thoughts that may help.

I'm running jclouds-1.8.0 using provider swift (auth-v1) writing to a swift
store.  In long test runs of my program (after writing 20-30k+ blobs of
approx 4MiB each), I will run into a flurry of failed writes (retries are
attempted, and fail), with a SocketTimeoutException "Read timed out"
reported.  There are no problems on the swift side.

I can't explain why the PUTs are failing.

The only correlation I can find is that the failures occur when the old
generation garbage collector runs.  I've tested this with CMS, G1,
ParallelGC, and in every instance, the failure occurrences occur with the
old gen runs.  Young gen runs don't seem to cause a problem, and each old
gen GC run doesn't take more than 1sec (and the timeouts I have configured
are all much longer than 1sec).

The jclouds code runs in a tomcat7 container and uses one instance of a
org.jclouds.blobstore.BlobStore.  I am using the strip-expect-header flag.

Thank you very very much for any ideas you have, either with the actual
problem or how to identify a root cause.

Aside: Number of JVM threads seems to go up staircase-wise with time, but
I'm not sure how to prove the threads are created by jclouds, and I'm not
sure if they are/will cause a problem or not.

Daniel Hsueh -- mailto:daniel.hsueh@seagate.com <daniel.hsueh@seagate.com>

View raw message