spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhideep Chakravarty <Abhideep.Chakrava...@mindtree.com>
Subject RE: Spark 1.2.0 not getting connected
Date Wed, 14 Jan 2015 06:09:50 GMT
Let me rephrase that it is a Play-Scala app which is trying to connect with the Spark.

From: Abhideep Chakravarty
Sent: Wednesday, January 14, 2015 11:34 AM
To: 'user@spark.apache.org'
Subject: Spark 1.2.0 not getting connected

Hi,

We are using Scala app to connect with the Spark and when we start the application we get
following error:

19:52:47.185 [sparkDriver-akka.actor.default-dispatcher-2] INFO  o.a.s.d.client.AppClient$ClientActor
- Connecting to master spark://172.22.193.138:7077...
19:52:47.198 [sparkDriver-akka.actor.default-dispatcher-5] WARN  Remoting - Tried to associate
with unreachable remote address [akka.tcp://sparkMaster@172.22.193.138:7077]. Address is now
gated for 5000 ms, all messages to this address will be delivered to dead letters. Reason:
Connection refused: /172.22.193.138:7077
19:52:47.200 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message Disassociated [akka.tcp://sparkDriver@172.22.193.138:54148] ->
[akka.tcp://sparkMaster@172.22.193.138:7077] from Actor[akka://sparkDriver/deadLetters]
19:52:47.201 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.0178 ms) Disassociated [akka.tcp://sparkDriver@172.22.193.138:54148]
-> [akka.tcp://sparkMaster@172.22.193.138:7077] from Actor[akka://sparkDriver/deadLetters]
19:52:47.202 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.d.client.AppClient$ClientActor
- [actor] received message AssociationError [akka.tcp://sparkDriver@172.22.193.138:54148]
-> [akka.tcp://sparkMaster@172.22.193.138:7077]: Error [Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077]
[
akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077
Caused by: akka.remote.transport.Transport$InvalidAssociationException: Connection refused:
/172.22.193.138:7077
] from Actor[akka://sparkDriver/deadLetters]
19:52:47.202 [sparkDriver-akka.actor.default-dispatcher-2] WARN  o.a.s.d.client.AppClient$ClientActor
- Could not connect to akka.tcp://sparkMaster@172.22.193.138:7077: akka.remote.InvalidAssociation:
Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077
19:52:47.202 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.d.client.AppClient$ClientActor
- [actor] handled message (0.5126 ms) AssociationError [akka.tcp://sparkDriver@172.22.193.138:54148]
-> [akka.tcp://sparkMaster@172.22.193.138:7077]: Error [Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077]
[
akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077
Caused by: akka.remote.transport.Transport$InvalidAssociationException: Connection refused:
/172.22.193.138:7077
] from Actor[akka://sparkDriver/deadLetters]
19:52:48.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:48.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3807 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:49.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:49.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3382 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:50.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:50.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2388 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:51.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:51.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.273601 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:52.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:52.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2757 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:53.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:53.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2631 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:54.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:54.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3165 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:55.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:55.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2841 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:56.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:56.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2209 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:57.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:57.136 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2898 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:58.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:58.136 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.4709 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:59.145 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:59.146 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3125 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:00.145 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:00.145 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2347 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:01.144 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:01.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.220302 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:02.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:02.146 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.278301 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:03.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:03.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.308 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:04.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:04.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.218 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:05.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:05.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2197 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:06.025 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.BlockManagerMasterActor
- [actor] received message ExpireDeadHosts from Actor[akka://sparkDriver/user/BlockManagerMaster#83889750]
19:53:06.026 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.BlockManagerMasterActor
- [actor] handled message (0.0909 ms) ExpireDeadHosts from Actor[akka://sparkDriver/user/BlockManagerMaster#83889750]
19:53:06.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:06.146 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2135 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:07.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:07.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2215 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:07.176 [sparkDriver-akka.actor.default-dispatcher-2] ERROR o.a.s.s.c.SparkDeploySchedulerBackend
- Application has been killed. Reason: All masters are unresponsive! Giving up.
19:53:07.179 [main] WARN  o.a.s.s.c.SparkDeploySchedulerBackend - Application ID is not initialized
yet.
19:53:07.181 [sparkDriver-akka.actor.default-dispatcher-2] ERROR o.a.s.scheduler.TaskSchedulerImpl
- Exiting due to error from cluster scheduler: All masters are unresponsive! Giving up.
19:53:07.186 [delete Spark temp dirs] DEBUG org.apache.spark.util.Utils - Shutdown hook called
19:53:07.192 [delete Spark local dirs] DEBUG o.a.spark.storage.DiskBlockManager - Shutdown
hook called


Can anyone suggest something ??

Regards,
Abhideep

________________________________

http://www.mindtree.com/email/disclaimer.html

Mime
View raw message