spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Play Scala Spark Exmaple
Date Sun, 11 Jan 2015 06:01:30 GMT
What is your spark version that is running on the EC2 cluster? From the build
file <https://github.com/knoldus/Play-Spark-Scala/blob/master/build.sbt> of
your play application it seems that it uses Spark 1.0.1.

Thanks
Best Regards

On Fri, Jan 9, 2015 at 7:17 PM, Eduardo Cusa <
eduardo.cusa@usmediaconsulting.com> wrote:

> Hi guys, I running the following example :
> https://github.com/knoldus/Play-Spark-Scala in the same machine as the
> spark master, and the spark cluster was lauched with ec2 script.
>
>
> I'm stuck with this errors, any idea how to fix it?
>
> Regards
> Eduardo
>
>
> call the play app prints the following exception :
>
>
> [*error*] a.r.EndpointWriter - AssociationError [akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481]
<- [akka.tcp://driverPropsFetcher@ip-10-158-18-250.ec2.internal:52575]: Error [Shut down
address: akka.tcp://driverPropsFetcher@ip-10-158-18-250.ec2.internal:52575] [
> akka.remote.ShutDownAssociation: Shut down address: akka.tcp://driverPropsFetcher@ip-10-158-18-250.ec2.internal:52575
> Caused by: akka.remote.transport.Transport$InvalidAssociationException: The remote system
terminated the association because it is shutting down.
>
>
>
>
> The master recive the spark application and generate the following stderr
> log page:
>
>
> 15/01/09 13:31:23 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutor@ip-10-158-18-250.ec2.internal:37856]
> 15/01/09 13:31:23 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkExecutor@ip-10-158-18-250.ec2.internal:37856]
> 15/01/09 13:31:23 INFO util.Utils: Successfully started service 'sparkExecutor' on port
37856.
> 15/01/09 13:31:23 INFO util.AkkaUtils: Connecting to MapOutputTracker: akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481/user/MapOutputTracker
> 15/01/09 13:31:23 INFO util.AkkaUtils: Connecting to BlockManagerMaster: akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481/user/BlockManagerMaster
> 15/01/09 13:31:23 INFO storage.DiskBlockManager: Created local directory at /mnt/spark/spark-local-20150109133123-3805
> 15/01/09 13:31:23 INFO storage.DiskBlockManager: Created local directory at /mnt2/spark/spark-local-20150109133123-b05e
> 15/01/09 13:31:23 INFO util.Utils: Successfully started service 'Connection manager for
block manager' on port 36936.
> 15/01/09 13:31:23 INFO network.ConnectionManager: Bound socket to port 36936 with id
= ConnectionManagerId(ip-10-158-18-250.ec2.internal,36936)
> 15/01/09 13:31:23 INFO storage.MemoryStore: MemoryStore started with capacity 265.4 MB
> 15/01/09 13:31:23 INFO storage.BlockManagerMaster: Trying to register BlockManager
> 15/01/09 13:31:23 INFO storage.BlockManagerMaster: Registered BlockManager
> 15/01/09 13:31:23 INFO util.AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481/user/HeartbeatReceiver
> 15/01/09 13:31:54 ERROR executor.CoarseGrainedExecutorBackend: Driver Disassociated [akka.tcp://sparkExecutor@ip-10-158-18-250.ec2.internal:57671]
-> [akka.tcp://sparkDriver@ip-10-28-236-122.ec2.internal:47481] disassociated! Shutting
down.
>
>

Mime
View raw message