mina-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Rathgeb (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SSHD-721) deadlock: all nio workers wait to be woken up
Date Mon, 05 Dec 2016 14:08:58 GMT
Markus Rathgeb created SSHD-721:
-----------------------------------

             Summary: deadlock: all nio workers wait to be woken up
                 Key: SSHD-721
                 URL: https://issues.apache.org/jira/browse/SSHD-721
             Project: MINA SSHD
          Issue Type: Bug
    Affects Versions: 1.3.0, 1.4.0
            Reporter: Markus Rathgeb


I am using sshd-core for a server machine (S) that accepts incoming connections and port forwarding
requests.
There are client machines (C) that run servers that should be accessible by a tunnel to the
server.
On the client machines (C) also an implementation using sshd-core is running that establish
the connection to the server (S) and initiate the port forwarding.

Other clients are using the tunnelled connection to communicate with the servers that are
running on the client machines (C).

Sometimes I realized that no data is transferred anymore (through the tunnels).

All the worker reside in the waitFor function and no one wakes them up.

{noformat}
"sshd-SshServer[67de991c]-nio2-thread-3" - Thread t@125
   java.lang.Thread.State: TIMED_WAITING
	at java.lang.Object.wait(Native Method)
	- waiting on <132c6b60> (a java.lang.Object)
	at org.apache.sshd.client.channel.AbstractClientChannel.waitFor(AbstractClientChannel.java:244)
	at org.apache.sshd.common.forward.DefaultTcpipForwarder$StaticIoHandler.messageReceived(DefaultTcpipForwarder.java:984)
	at org.apache.sshd.common.io.nio2.Nio2Session.handleReadCycleCompletion(Nio2Session.java:276)
	at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:256)
	at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:253)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$$Lambda$45/1071326492.run(Unknown
Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126)
	at sun.nio.ch.Invoker$2.run(Invoker.java:218)
	at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
	- locked <6d02d7ed> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"sshd-SshServer[67de991c]-nio2-thread-2" - Thread t@124
   java.lang.Thread.State: TIMED_WAITING
	at java.lang.Object.wait(Native Method)
	- waiting on <7e9f4eff> (a java.lang.Object)
	at org.apache.sshd.client.channel.AbstractClientChannel.waitFor(AbstractClientChannel.java:244)
	at org.apache.sshd.common.forward.DefaultTcpipForwarder$StaticIoHandler.messageReceived(DefaultTcpipForwarder.java:984)
	at org.apache.sshd.common.io.nio2.Nio2Session.handleReadCycleCompletion(Nio2Session.java:276)
	at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:256)
	at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:253)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$$Lambda$45/1071326492.run(Unknown
Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126)
	at sun.nio.ch.Invoker$2.run(Invoker.java:218)
	at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
	- locked <35fbf3e8> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"sshd-SshServer[67de991c]-nio2-thread-1" - Thread t@122
   java.lang.Thread.State: TIMED_WAITING
	at java.lang.Object.wait(Native Method)
	- waiting on <49ce93a9> (a java.lang.Object)
	at org.apache.sshd.client.channel.AbstractClientChannel.waitFor(AbstractClientChannel.java:244)
	at org.apache.sshd.common.forward.DefaultTcpipForwarder$StaticIoHandler.messageReceived(DefaultTcpipForwarder.java:984)
	at org.apache.sshd.common.io.nio2.Nio2Session.handleReadCycleCompletion(Nio2Session.java:276)
	at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:256)
	at org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:253)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$$Lambda$45/1071326492.run(Unknown
Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126)
	at sun.nio.ch.Invoker.invokeDirect(Invoker.java:157)
	at sun.nio.ch.UnixAsynchronousSocketChannelImpl.implRead(UnixAsynchronousSocketChannelImpl.java:553)
	at sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:276)
	at sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:297)
	at org.apache.sshd.common.io.nio2.Nio2Session.doReadCycle(Nio2Session.java:304)
	at org.apache.sshd.common.io.nio2.Nio2Session.doReadCycle(Nio2Session.java:249)
	at org.apache.sshd.common.io.nio2.Nio2Session.startReading(Nio2Session.java:243)
	at org.apache.sshd.common.io.nio2.Nio2Session.startReading(Nio2Session.java:239)
	at org.apache.sshd.common.io.nio2.Nio2Session.startReading(Nio2Session.java:235)
	at org.apache.sshd.common.io.nio2.Nio2Session.startReading(Nio2Session.java:231)
	at org.apache.sshd.common.io.nio2.Nio2Session.startReading(Nio2Session.java:227)
	at org.apache.sshd.common.io.nio2.Nio2Acceptor$AcceptCompletionHandler.onCompleted(Nio2Acceptor.java:178)
	at org.apache.sshd.common.io.nio2.Nio2Acceptor$AcceptCompletionHandler.onCompleted(Nio2Acceptor.java:156)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler$$Lambda$45/1071326492.run(Unknown
Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126)
	at sun.nio.ch.Invoker$2.run(Invoker.java:218)
	at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
	- locked <7d1c59e6> (a java.util.concurrent.ThreadPoolExecutor$Worker)

"sshd-SshServer[67de991c]-timer-thread-1" - Thread t@105
   java.lang.Thread.State: TIMED_WAITING
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for <655c080c> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
- None
{noformat}

To eliminate some other code that could trigger that error I created a "minimal" example --
a simple test application -- that could be used to demonstrate the hang (for me it is reproducible
using that code).

Please have a look at https://github.com/maggu2810/sshd-deadlock/tree/first-report where you
could also find a readme with a short description about the code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message