spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhideep Chakravarty <Abhideep.Chakrava...@mindtree.com>
Subject RE: Spark 1.2.0 not getting connected
Date Wed, 14 Jan 2015 09:19:39 GMT
Hi Akhil,

[cid:image001.png@01D03008.40E75B40]


Config:
scala-spark-jar-location=””
spark-master-location="spark://172.22.193.138:7007"

Error log msg:
14:46:52.443 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Ac
tor[akka://sparkDriver/user/CoarseGrainedScheduler#-1386691586]
14:46:52.483 [sparkDriver-akka.actor.default-dispatcher-3] INFO  o.a.s.d.client.AppClient$ClientActor
- Connecting to master spark://172.22.193.138:70
07...
14:46:52.491 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (39.773252 ms) Revive
Offers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1386691586]
14:46:53.185 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message Disassociated [akka.
tcp://sparkDriver@A2ML11326.mindtree.com:55617] -> [akka.tcp://sparkMaster@172.22.193.138:7007]
from Actor[akka://sparkDriver/deadLetters]
14:46:53.193 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (2.050513 ms) Disasso
ciated [akka.tcp://sparkDriver@A2ML11326.mindtree.com:55617] -> [akka.tcp://sparkMaster@172.22.193.138:7007]
from Actor[akka://sparkDriver/deadLetters
]
14:46:53.201 [sparkDriver-akka.actor.default-dispatcher-3] WARN  a.remote.ReliableDeliverySupervisor
- Association with remote system [akka.tcp://spar
kMaster@172.22.193.138:7007<mailto:kMaster@172.22.193.138:7007>] has failed, address
is now gated for [5000] ms. Reason is: [Disassociated].
14:46:53.207 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message Disassociated [akka.
tcp://sparkDriver@A2ML11326.mindtree.com:55617] -> [akka.tcp://sparkMaster@172.22.193.138:7007]
from Actor[akka://sparkDriver/deadLetters]
14:46:53.208 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.015172 ms) Disasso
ciated [akka.tcp://sparkDriver@A2ML11326.mindtree.com:55617] -> [akka.tcp://sparkMaster@172.22.193.138:7007]
from Actor[akka://sparkDriver/deadLetters
]
14:46:53.436 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Ac
tor[akka://sparkDriver/user/CoarseGrainedScheduler#-1386691586]
14:46:53.437 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.420366 ms) ReviveO
ffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1386691586]
14:46:54.444 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Ac
tor[akka://sparkDriver/user/CoarseGrainedScheduler#-1386691586]




From: Akhil Das [mailto:akhil@sigmoidanalytics.com]
Sent: Wednesday, January 14, 2015 12:39 PM
To: Abhideep Chakravarty
Cc: user@spark.apache.org
Subject: Re: Spark 1.2.0 not getting connected

Its clearly saying Connection refused: 172.22.193.138:7077<http://172.22.193.138:7077>
just make sure your master URL is listed as spark://172.22.193.138:7077<http://172.22.193.138:7077>
in the webUI(that running on port 8080), also be sure that no firewall/network is blocking
the connection (simple telnet 172.22.193.138 7077 will work if there's no network issue)

Thanks
Best Regards

On Wed, Jan 14, 2015 at 11:39 AM, Abhideep Chakravarty <Abhideep.Chakravarty@mindtree.com<mailto:Abhideep.Chakravarty@mindtree.com>>
wrote:
Let me rephrase that it is a Play-Scala app which is trying to connect with the Spark.

From: Abhideep Chakravarty
Sent: Wednesday, January 14, 2015 11:34 AM
To: 'user@spark.apache.org<mailto:user@spark.apache.org>'
Subject: Spark 1.2.0 not getting connected

Hi,

We are using Scala app to connect with the Spark and when we start the application we get
following error:

19:52:47.185 [sparkDriver-akka.actor.default-dispatcher-2] INFO  o.a.s.d.client.AppClient$ClientActor
- Connecting to master spark://172.22.193.138:7077...
19:52:47.198 [sparkDriver-akka.actor.default-dispatcher-5] WARN  Remoting - Tried to associate
with unreachable remote address [akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>].
Address is now gated for 5000 ms, all messages to this address will be delivered to dead letters.
Reason: Connection refused: /172.22.193.138:7077<http://172.22.193.138:7077>
19:52:47.200 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message Disassociated [akka.tcp://sparkDriver@172.22.193.138:54148<http://sparkDriver@172.22.193.138:54148>]
-> [akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>]
from Actor[akka://sparkDriver/deadLetters]
19:52:47.201 [sparkDriver-akka.actor.default-dispatcher-3] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.0178 ms) Disassociated [akka.tcp://sparkDriver@172.22.193.138:54148<http://sparkDriver@172.22.193.138:54148>]
-> [akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>]
from Actor[akka://sparkDriver/deadLetters]
19:52:47.202 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.d.client.AppClient$ClientActor
- [actor] received message AssociationError [akka.tcp://sparkDriver@172.22.193.138:54148<http://sparkDriver@172.22.193.138:54148>]
-> [akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>]:
Error [Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>]
[
akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>
Caused by: akka.remote.transport.Transport$InvalidAssociationException: Connection refused:
/172.22.193.138:7077<http://172.22.193.138:7077>
] from Actor[akka://sparkDriver/deadLetters]
19:52:47.202 [sparkDriver-akka.actor.default-dispatcher-2] WARN  o.a.s.d.client.AppClient$ClientActor
- Could not connect to akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>:
akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>
19:52:47.202 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.d.client.AppClient$ClientActor
- [actor] handled message (0.5126 ms) AssociationError [akka.tcp://sparkDriver@172.22.193.138:54148<http://sparkDriver@172.22.193.138:54148>]
-> [akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>]:
Error [Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>]
[
akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@172.22.193.138:7077<http://sparkMaster@172.22.193.138:7077>
Caused by: akka.remote.transport.Transport$InvalidAssociationException: Connection refused:
/172.22.193.138:7077<http://172.22.193.138:7077>
] from Actor[akka://sparkDriver/deadLetters]
19:52:48.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:48.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3807 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:49.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:49.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3382 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:50.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:50.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2388 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:51.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:51.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.273601 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:52.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:52.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2757 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:53.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:53.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2631 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:54.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:54.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3165 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:55.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:55.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2841 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:56.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:56.136 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2209 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:57.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:57.136 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2898 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:58.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:58.136 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.4709 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:59.145 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:52:59.146 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.3125 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:00.145 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:00.145 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2347 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:01.144 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:01.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.220302 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:02.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:02.146 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.278301 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:03.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:03.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.308 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:04.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:04.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.218 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:05.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:05.135 [sparkDriver-akka.actor.default-dispatcher-15] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2197 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:06.025 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.BlockManagerMasterActor
- [actor] received message ExpireDeadHosts from Actor[akka://sparkDriver/user/BlockManagerMaster#83889750]
19:53:06.026 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.BlockManagerMasterActor
- [actor] handled message (0.0909 ms) ExpireDeadHosts from Actor[akka://sparkDriver/user/BlockManagerMaster#83889750]
19:53:06.145 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:06.146 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2135 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:07.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] received message ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:07.135 [sparkDriver-akka.actor.default-dispatcher-2] DEBUG o.a.s.s.c.SparkDeploySchedulerBackend
- [actor] handled message (0.2215 ms) ReviveOffers from Actor[akka://sparkDriver/user/CoarseGrainedScheduler#-1468085206]
19:53:07.176 [sparkDriver-akka.actor.default-dispatcher-2] ERROR o.a.s.s.c.SparkDeploySchedulerBackend
- Application has been killed. Reason: All masters are unresponsive! Giving up.
19:53:07.179 [main] WARN  o.a.s.s.c.SparkDeploySchedulerBackend - Application ID is not initialized
yet.
19:53:07.181 [sparkDriver-akka.actor.default-dispatcher-2] ERROR o.a.s.scheduler.TaskSchedulerImpl
- Exiting due to error from cluster scheduler: All masters are unresponsive! Giving up.
19:53:07.186 [delete Spark temp dirs] DEBUG org.apache.spark.util.Utils - Shutdown hook called
19:53:07.192 [delete Spark local dirs] DEBUG o.a.spark.storage.DiskBlockManager - Shutdown
hook called


Can anyone suggest something ??

Regards,
Abhideep

________________________________

http://www.mindtree.com/email/disclaimer.html

Mime
View raw message