jclouds-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Gaul <g...@apache.org>
Subject Re: java.net.SocketTimeoutException correlated with old generation GC
Date Thu, 04 Dec 2014 15:26:17 GMT
[Sorry I missed this earlier due to some extended travel.]

Can you show us the exact code for your putBlob calls?  Have you tried
attaching a profiler like YourKit or tool like file-leak-detector to
your application?  Thread, socket, or some other file descriptor leaks
can cause weird behavior with the GC and finalizer which can manifest in
the application.  We have one report of unbounded connection use in
JCLOUDS_628:

https://issues.apache.org/jira/browse/JCLOUDS-628

On Mon, Nov 17, 2014 at 02:49:05PM -0500, Daniel Hsueh wrote:
> Hello,
> 
> I'm experiencing behavior I can't explain and I'm wondering if you have any
> thoughts that may help.
> 
> I'm running jclouds-1.8.0 using provider swift (auth-v1) writing to a swift
> store.  In long test runs of my program (after writing 20-30k+ blobs of
> approx 4MiB each), I will run into a flurry of failed writes (retries are
> attempted, and fail), with a SocketTimeoutException "Read timed out"
> reported.  There are no problems on the swift side.
> 
> I can't explain why the PUTs are failing.
> 
> The only correlation I can find is that the failures occur when the old
> generation garbage collector runs.  I've tested this with CMS, G1,
> ParallelGC, and in every instance, the failure occurrences occur with the
> old gen runs.  Young gen runs don't seem to cause a problem, and each old
> gen GC run doesn't take more than 1sec (and the timeouts I have configured
> are all much longer than 1sec).
> 
> The jclouds code runs in a tomcat7 container and uses one instance of a
> org.jclouds.blobstore.BlobStore.  I am using the strip-expect-header flag.
> 
> Thank you very very much for any ideas you have, either with the actual
> problem or how to identify a root cause.
> 
> Aside: Number of JVM threads seems to go up staircase-wise with time, but
> I'm not sure how to prove the threads are created by jclouds, and I'm not
> sure if they are/will cause a problem or not.
> 
> -- 
> Daniel Hsueh -- mailto:daniel.hsueh@seagate.com <daniel.hsueh@seagate.com>
> tel:+1-905-287-2167

-- 
Andrew Gaul
http://gaul.org/

Mime
View raw message