storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Victor Kovrizhkin <vik.kovrizh...@gmail.com>
Subject Re: Storm 1.0.0 DRPC connection refused
Date Tue, 19 Apr 2016 08:04:17 GMT
Please help! 

From:  Victor Kovrizhkin
Date:  Monday, April 18, 2016 at 9:28 PM
To:  "user@storm.apache.org"
Subject:  Storm 1.0.0 DRPC connection refused 

Hi Good People!

I’m trying to update my cluster running Storm 0.10.0 with DRPC to Storm 1.0.0. I’ve updated
all machines with latest version of Storm, changed storm.yml configurations (e.g. nimbus.host
-> nimbus.seeds), changed dependencies in topology. When I starting nimbus, supervisors,
ui, drpc, logviewers everything looks good, I’m able to see UI, logs, configuration and
submit topologies. 

But once I submit DRPC topology – it’s logs full of following exceptions: 

java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
org.apache.storm.thrift.transport.TTransportException: java.net.ConnectException: Connection
refused

at org.apache.storm.drpc.DRPCSpout.checkFutures(DRPCSpout.java:129) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCSpout.nextTuple(DRPCSpout.java:210) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.daemon.executor$fn__8158$fn__8173$fn__8204.invoke(executor.clj:647) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.util$async_loop$fn__554.invoke(util.clj:484) [storm-core-1.0.0.jar:1.0.0]

at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_74]

Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused

at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_74]

at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_74]

at org.apache.storm.drpc.DRPCSpout.checkFutures(DRPCSpout.java:127) ~[storm-core-1.0.0.jar:1.0.0]

... 5 more

Caused by: java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused

at org.apache.storm.security.auth.TBackoffConnect.retryNext(TBackoffConnect.java:64) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:56)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:99) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:46) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCSpout$Adder.call(DRPCSpout.java:101) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCSpout$Adder.call(DRPCSpout.java:88) ~[storm-core-1.0.0.jar:1.0.0]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_74]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_74]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_74]

... 1 more

Caused by: org.apache.storm.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused

at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:226) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:53)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:99) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:46) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCSpout$Adder.call(DRPCSpout.java:101) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCSpout$Adder.call(DRPCSpout.java:88) ~[storm-core-1.0.0.jar:1.0.0]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_74]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_74]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_74]

... 1 more

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_74]

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_74]

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:204) ~[?:1.8.0_74]

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_74]

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_74]

at java.net.Socket.connect(Socket.java:589) ~[?:1.8.0_74]

at org.apache.storm.thrift.transport.TSocket.open(TSocket.java:221) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.thrift.transport.TFramedTransport.open(TFramedTransport.java:81) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.SimpleTransportPlugin.connect(SimpleTransportPlugin.java:103)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.TBackoffConnect.doConnectWithRetry(TBackoffConnect.java:53)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.reconnect(ThriftClient.java:99) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:69) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.security.auth.ThriftClient.<init>(ThriftClient.java:46) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCInvocationsClient.<init>(DRPCInvocationsClient.java:40)
~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCSpout$Adder.call(DRPCSpout.java:101) ~[storm-core-1.0.0.jar:1.0.0]

at org.apache.storm.drpc.DRPCSpout$Adder.call(DRPCSpout.java:88) ~[storm-core-1.0.0.jar:1.0.0]

at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_74]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_74]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_74]

... 1 more

2016-04-18 18:10:19.579 o.a.s.util [ERROR]  Async loop died!


It seems DRPCSpout is unable to connect to DRPC server in some reasons. 
Everything is fine with Storm 0.10.0
Supervisor’s configuration is correct – there is valid ‘drpc.servers’ entry.
Zookeeper and Nimbus are running
I’m able to telnet DRPC server 3772, 3774, 3773 ports from any supervisor. 
DRPC server is up and running, it is able to handle request (with timeout of course), here
is the log: 
2016-04-18 18:00:20.444 o.a.s.d.drpc [INFO] Starting drpc server for storm version '1.0.0'

2016-04-18 18:00:20.619 o.a.s.d.drpc [INFO] Starting Distributed RPC servers...

2016-04-18 18:00:20.692 o.a.s.s.o.e.j.s.Server [INFO] jetty-7.x.y-SNAPSHOT

2016-04-18 18:00:20.717 o.a.s.s.o.e.j.s.h.ContextHandler [INFO] started o.a.s.s.o.e.j.s.ServletContextHandler{/,null}

2016-04-18 18:00:20.728 o.a.s.s.o.e.j.s.AbstractConnector [INFO] Started SelectChannelConnector@0.0.0.0:3774

2016-04-18 18:00:20.729 o.a.s.d.m.MetricsUtils [INFO] Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter

2016-04-18 18:00:20.730 o.a.s.d.m.r.JmxPreparableReporter [INFO] Preparing...

2016-04-18 18:00:20.737 o.a.s.d.common [INFO] Started statistics report plugin...

2016-04-18 18:09:49.790 o.a.s.t.s.AbstractNonblockingServer$FrameBuffer [ERROR] Read an invalid
frame size of -786435. Are you using TFramedTransport on the client side?

2016-04-18 18:26:10.413 o.a.s.d.drpc [INFO] Starting drpc server for storm version '1.0.0'

2016-04-18 18:26:10.743 o.a.s.d.drpc [INFO] Starting Distributed RPC servers...

2016-04-18 18:26:10.852 o.a.s.s.o.e.j.s.Server [INFO] jetty-7.x.y-SNAPSHOT

2016-04-18 18:26:10.896 o.a.s.s.o.e.j.s.h.ContextHandler [INFO] started o.a.s.s.o.e.j.s.ServletContextHandler{/,null}

2016-04-18 18:26:10.921 o.a.s.s.o.e.j.s.AbstractConnector [INFO] Started SelectChannelConnector@0.0.0.0:3774

2016-04-18 18:26:10.922 o.a.s.d.m.MetricsUtils [INFO] Using statistics reporter plugin:org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter

2016-04-18 18:26:10.924 o.a.s.d.m.r.JmxPreparableReporter [INFO] Preparing...

2016-04-18 18:26:10.935 o.a.s.d.common [INFO] Started statistics report plugin...


Please help
Sincerely,
Viktor 


Mime
View raw message