nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff <jtsw...@gmail.com>
Subject Re: Phantom node
Date Thu, 04 May 2017 01:27:47 GMT
Can you provide some information on the configuration (nifi.properties) of
the nodes in your cluster?  Can each node in your cluster ping all the
other nodes?  Are you running embedded ZooKeeper, or an external one?

On Wed, May 3, 2017 at 8:11 PM Neil Derraugh <
neil.derraugh@intellifylearning.com> wrote:

> I can't load the canvas right now on our cluster.  I get this error from
> one of the nodes nifi-app.logs
>
> 2017-05-03 23:40:30,207 WARN [Replicate Request Thread-2]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
> /nifi-api/flow/current-user to 10.80.53.39:31212 due to {}
> com.sun.jersey.api.client.ClientHandlerException:
> java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> ~[jersey-client-1.19.jar:1.19]
> at
> com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
> ~[jersey-client-1.19.jar:1.19]
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:579)
> ~[nifi-framework-cluster-1.1.2.jar:1.1.2]
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:771)
> ~[nifi-framework-cluster-1.1.2.jar:1.1.2]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: java.net.NoRouteToHostException: Host is unreachable (Host
> unreachable)
> at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_121]
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> ~[na:1.8.0_121]
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> ~[na:1.8.0_121]
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> ~[na:1.8.0_121]
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> ~[na:1.8.0_121]
> at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_121]
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
> ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
> ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.New(HttpClient.java:308) ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.New(HttpClient.java:326) ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
> ~[na:1.8.0_121]
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> ~[na:1.8.0_121]
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
> ~[jersey-client-1.19.jar:1.19]
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
> ~[jersey-client-1.19.jar:1.19]
> ... 12 common frames omitted
> 2017-05-03 23:40:30,207 WARN [Replicate Request Thread-2]
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> com.sun.jersey.api.client.ClientHandlerException:
> java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> ~[jersey-client-1.19.jar:1.19]
> at
> com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
> ~[jersey-client-1.19.jar:1.19]
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:579)
> ~[nifi-framework-cluster-1.1.2.jar:1.1.2]
> at
> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:771)
> ~[nifi-framework-cluster-1.1.2.jar:1.1.2]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_121]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
> Caused by: java.net.NoRouteToHostException: Host is unreachable (Host
> unreachable)
> at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_121]
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> ~[na:1.8.0_121]
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> ~[na:1.8.0_121]
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> ~[na:1.8.0_121]
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> ~[na:1.8.0_121]
> at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_121]
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
> ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
> ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.New(HttpClient.java:308) ~[na:1.8.0_121]
> at sun.net.www.http.HttpClient.New(HttpClient.java:326) ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1202)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1138)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1032)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:966)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1546)
> ~[na:1.8.0_121]
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
> ~[na:1.8.0_121]
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> ~[na:1.8.0_121]
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
> ~[jersey-client-1.19.jar:1.19]
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
> ~[jersey-client-1.19.jar:1.19]
> ... 12 common frames omitted
>
>
> GETing the nifi-api's controller/cluster I get the list of happy nodes and
> then this one (IP address changed to protect the innocent):
> {
> nodeId: "7f7c1a9e-faa6-413b-9317-bcec4996cb14",
> address: "w.x.y.z",
> apiPort: 31212,
> status: "CONNECTED",
> roles: [ ],
> events: [ ]
> }
>
> No events, no roles.  It says it's connected but has no heartbeat and it's
> not part of the list of running jobs so far as I can detect.  It's likely a
> node that had previously been a health member of the cluster.
>
> Can anybody help me interpret this?
>
> I deleted the node and am carrying on as usual.  Just wondering if anyone
> has any insight into why it would leave the node in the cluster and show it
> as connected.
>
> Thanks,
> Neil
>

Mime
View raw message