spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eduardo Cusa <eduardo.c...@usmediaconsulting.com>
Subject Play Scala Spark Exmaple
Date Fri, 09 Jan 2015 13:47:12 GMT
Hi guys, I running the following example :
https://github.com/knoldus/Play-Spark-Scala in the same machine as the
spark master, and the spark cluster was lauched with ec2 script.


I'm stuck with this errors, any idea how to fix it?

Regards
Eduardo


call the play app prints the following exception :


[*error*] a.r.EndpointWriter - AssociationError
[akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481] <-
[akka.tcp://driverPropsFetcher@ip-10-158-18-250.ec2.internal:52575]:
Error [Shut down address:
akka.tcp://driverPropsFetcher@ip-10-158-18-250.ec2.internal:52575] [
akka.remote.ShutDownAssociation: Shut down address:
akka.tcp://driverPropsFetcher@ip-10-158-18-250.ec2.internal:52575
Caused by: akka.remote.transport.Transport$InvalidAssociationException:
The remote system terminated the association because it is shutting
down.




The master recive the spark application and generate the following stderr
log page:


15/01/09 13:31:23 INFO Remoting: Remoting started; listening on
addresses :[akka.tcp://sparkExecutor@ip-10-158-18-250.ec2.internal:37856]
15/01/09 13:31:23 INFO Remoting: Remoting now listens on addresses:
[akka.tcp://sparkExecutor@ip-10-158-18-250.ec2.internal:37856]
15/01/09 13:31:23 INFO util.Utils: Successfully started service
'sparkExecutor' on port 37856.
15/01/09 13:31:23 INFO util.AkkaUtils: Connecting to MapOutputTracker:
akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481/user/MapOutputTracker
15/01/09 13:31:23 INFO util.AkkaUtils: Connecting to
BlockManagerMaster:
akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481/user/BlockManagerMaster
15/01/09 13:31:23 INFO storage.DiskBlockManager: Created local
directory at /mnt/spark/spark-local-20150109133123-3805
15/01/09 13:31:23 INFO storage.DiskBlockManager: Created local
directory at /mnt2/spark/spark-local-20150109133123-b05e
15/01/09 13:31:23 INFO util.Utils: Successfully started service
'Connection manager for block manager' on port 36936.
15/01/09 13:31:23 INFO network.ConnectionManager: Bound socket to port
36936 with id =
ConnectionManagerId(ip-10-158-18-250.ec2.internal,36936)
15/01/09 13:31:23 INFO storage.MemoryStore: MemoryStore started with
capacity 265.4 MB
15/01/09 13:31:23 INFO storage.BlockManagerMaster: Trying to register
BlockManager
15/01/09 13:31:23 INFO storage.BlockManagerMaster: Registered BlockManager
15/01/09 13:31:23 INFO util.AkkaUtils: Connecting to
HeartbeatReceiver:
akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481/user/HeartbeatReceiver
15/01/09 13:31:54 ERROR executor.CoarseGrainedExecutorBackend: Driver
Disassociated [akka.tcp://sparkExecutor@ip-10-158-18-250.ec2.internal:57671]
-> [akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481]
disassociated! Shutting down.

Mime
View raw message