spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nancy henry <nancyhenry6...@gmail.com>
Subject keep or remove sc.stop() coz of RpcEnv already stopped error
Date Mon, 13 Mar 2017 11:08:32 GMT
 Hi Team,


getting this error if we put sc.stop() in application..



can we remove it from application but i read if yu dont explicitly stop
using sc.stop the yarn application will not get registered in history
service.. SO what to do ?


 WARN Dispatcher: Message RemoteProcessDisconnected dropped.

java.lang.IllegalStateException: RpcEnv already stopped.

        at
org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:159)

        at
org.apache.spark.rpc.netty.Dispatcher.postToAll(Dispatcher.scala:109)

        at
org.apache.spark.rpc.netty.NettyRpcHandler.connectionTerminated(NettyRpcEnv.scala:630)

        at
org.apache.spark.network.server.TransportRequestHandler.channelUnregistered(TransportRequestHandler.java:94)

Mime
View raw message