spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aseem Bansal <asmbans...@gmail.com>
Subject spark keeps on creating executors and each one fails with "TransportClient has not yet been set."
Date Thu, 02 Mar 2017 12:04:15 GMT
Is there a way to find out what is it trying to connect to? I am running my
spark client from within a docker container so I opened up various ports as
per
http://stackoverflow.com/questions/27729010/how-to-configure-apache-spark-random-worker-ports-for-tight-firewalls
after adding all the properties in conf/spark-defaults.conf on my spark
cluster's installation.

In stdout of the executor I can see the following debug

17/03/02 12:01:17 DEBUG UserGroupInformation: PrivilegedActionException
as:root (auth:SIMPLE) cause:org.apache.spark.rpc.RpcTimeoutException:
Cannot receive any reply in 120 seconds. This timeout is controlled by
spark.rpc.askTimeout

What is it trying to connect to?

Mime
View raw message