sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1916) Sqoop2: Yarn child leaking in integration tests
Date Mon, 07 Mar 2016 17:31:40 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15183314#comment-15183314
] 

Jarek Jarcec Cecho commented on SQOOP-1916:
-------------------------------------------

It's interesting that you see the leaking children on real cluster as well [~skuehn]. We've
observed them only in the integration tests that are running on mini clusters. Have you by
any chance tried the latest trunk as well? The trunk has significantly moved from last released
version, so I'm wondering if you can still observe the behavior.

> Sqoop2: Yarn child leaking in integration tests
> -----------------------------------------------
>
>                 Key: SQOOP-1916
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1916
>             Project: Sqoop
>          Issue Type: Bug
>          Components: sqoop2-test
>            Reporter: Abraham Elmahrek
>            Assignee: Scott Kuehn
>             Fix For: 2.0.0
>
>         Attachments: SQOOP-1916.patch
>
>
> It seems like the yarn child processes are leaking.
> {noformat:title=jstack}
>  jstack 57054
> 2014-12-16 21:39:06
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode):
> "Attach Listener" daemon prio=5 tid=0x00007f90ebd57000 nid=0x610f
> waiting on condition [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
> "DestroyJavaVM" prio=5 tid=0x00007f90ebd42800 nid=0x1903 waiting on
> condition [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
> "Abandoned connection cleanup thread" daemon prio=5
> tid=0x00007f90ebce8800 nid=0x7103 in Object.wait()
> [0x00000001129c9000]
>    java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000007fd3b9668> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x00000007fd3b9668> (a java.lang.ref.ReferenceQueue$Lock)
> at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:41)
> "OutputFormatLoader-consumer" prio=5 tid=0x00007f90eb9b5800 nid=0x6d03
> waiting on condition [0x00000001127c3000]
>    java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0x00000007fc9e5708> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> "org.apache.hadoop.hdfs.PeerCache@2582a699" daemon prio=5
> tid=0x00007f90eca20000 nid=0x6b03 waiting on condition
> [0x00000001126c0000]
>    java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:244)
> at org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:41)
> at org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:119)
> at java.lang.Thread.run(Thread.java:745)
> "Service Thread" daemon prio=5 tid=0x00007f90ec81f800 nid=0x5303
> runnable [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread1" daemon prio=5 tid=0x00007f90ef001000 nid=0x5103
> waiting on condition [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
> "C2 CompilerThread0" daemon prio=5 tid=0x00007f90ed819000 nid=0x4f03
> waiting on condition [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
> "Signal Dispatcher" daemon prio=5 tid=0x00007f90ed818000 nid=0x4d03
> runnable [0x0000000000000000]
>    java.lang.Thread.State: RUNNABLE
> "Finalizer" daemon prio=5 tid=0x00007f90ec821800 nid=0x3903 in
> Object.wait() [0x0000000110ba2000]
>    java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000007ff842558> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
> - locked <0x00000007ff842558> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
> "Reference Handler" daemon prio=5 tid=0x00007f90eb811800 nid=0x3703 in
> Object.wait() [0x0000000110a9f000]
>    java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000007ff841f40> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:503)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
> - locked <0x00000007ff841f40> (a java.lang.ref.Reference$Lock)
> "VM Thread" prio=5 tid=0x00007f90eb80f000 nid=0x3503 runnable
> "GC task thread#0 (ParallelGC)" prio=5 tid=0x00007f90ec00e000
> nid=0x2503 runnable
> "GC task thread#1 (ParallelGC)" prio=5 tid=0x00007f90ec00f000
> nid=0x2703 runnable
> "GC task thread#2 (ParallelGC)" prio=5 tid=0x00007f90eb80d000
> nid=0x2903 runnable
> "GC task thread#3 (ParallelGC)" prio=5 tid=0x00007f90eb80e000
> nid=0x2b03 runnable
> "GC task thread#4 (ParallelGC)" prio=5 tid=0x00007f90ec00f800
> nid=0x2d03 runnable
> "GC task thread#5 (ParallelGC)" prio=5 tid=0x00007f90ec800800
> nid=0x2f03 runnable
> "GC task thread#6 (ParallelGC)" prio=5 tid=0x00007f90ed000000
> nid=0x3103 runnable
> "GC task thread#7 (ParallelGC)" prio=5 tid=0x00007f90eb80e800
> nid=0x3303 runnable
> "VM Periodic Task Thread" prio=5 tid=0x00007f90ed82a800 nid=0x5503
> waiting on condition
> JNI global references: 184
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message