spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Agrawal <saurabh.agra...@markit.com>
Subject ALS train error
Date Tue, 25 Nov 2014 14:14:39 GMT

Hi,

I am getting the following error

val model = ALS.train(ratings, rank, numIterations, 0.01)

org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 103.0 failed
1 times, most recent failure: Lost task 1.0 in stage 103.0 (TID 3, localhost): scala.MatchError:
[Ljava.lang.String;@4837e797 (of class [Ljava.lang.String;)
        $iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:16)
        $iwC$$iwC$$iwC$$iwC$$anonfun$1.apply(<console>:16)
        scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
        scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
        scala.collection.Iterator$class.foreach(Iterator.scala:727)
        scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
        org.apache.spark.shuffle.hash.HashShuffleWriter.write(HashShuffleWriter.scala:65)
        org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:68)
        org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
        org.apache.spark.scheduler.Task.run(Task.scala:54)
        org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
        java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        java.lang.Thread.run(Thread.java:745)
Driver stacktrace:
                at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185)
                at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174)
                at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173)
                at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
                at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
                at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173)
                at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
                at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688)
                at scala.Option.foreach(Option.scala:236)
                at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688)
                at org.apache.spark.scheduler.DAGSchedulerEventProcessActor$$anonfun$receive$2.applyOrElse(DAGScheduler.scala:1391)
                at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498)
                at akka.actor.ActorCell.invoke(ActorCell.scala:456)
                at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237)
                at akka.dispatch.Mailbox.run(Mailbox.scala:219)
                at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386)
                at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
                at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
                at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
                at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Thanks!!

Regards,
Saurabh Agrawal


________________________________
This e-mail, including accompanying communications and attachments, is strictly confidential
and only for the intended recipient. Any retention, use or disclosure not expressly authorised
by Markit is prohibited. This email is subject to all waivers and other terms at the following
link: http://www.markit.com/en/about/legal/email-disclaimer.page

Please visit http://www.markit.com/en/about/contact/contact-us.page? for contact information
on our offices worldwide.

MarkitSERV Limited has its registered office located at Level 4, Ropemaker Place, 25 Ropemaker
Street, London, EC2Y 9LY and is authorized and regulated by the Financial Conduct Authority
with registration number 207294

Mime
View raw message