mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pat Ferrel <...@occamsmachete.com>
Subject Re: No configuration setting found for key 'akka.event-handlers' when running spark-itemsimilarity
Date Mon, 22 Dec 2014 17:32:58 GMT
For local mode the job is running with the bits it was linked to. Make sure these are the same
bits running on your cluster. Mahout links to Spark 1.1.0, is that what you are running on
the cluster?

On Dec 21, 2014, at 9:06 PM, hlqv <hlqvuong@gmail.com> wrote:

@Pat Ferrel: Thank you so much. I restarted Hadoop and now it works fine
for local mode (--master local[n]).
I ran an example JavaWordCount with --master spark://localhost:7077 and got
the correct result. But when I ran spark-itemsimilarity again with --master
spark://localhost:7077 then I got the error in the log:
ERROR SparkDeploySchedulerBackend: Application has been killed. Reason: All
masters are unresponsive! Giving up
It seems relating to the WARN log:
Initial job has not accepted any resources; check your cluster UI to ensure
that workers are registered and have sufficient memory

I have looked the Spark UI, every thing look correct with two workers are
available and 4GB RAM is usable.

On 22 December 2014 at 06:11, Pat Ferrel <pat@occamsmachete.com> wrote:

> Looks like it's refusing to initialize your spark context. Did you start
> the master spark://localhost:7077 before running mahout? Can you see a UI
> at localhost:8080?
> 
> If no to any of those try  --master local[n], which will run the job
> locally not in a single machine cluster and will run much faster. Replace n
> with the number of cores you want to allocate, like local[7].
> 
> You should use the Spark start scripts when using a single machine cluster.
> 
> If none of this helps, are you able to run any of the Spark examples on
> spark://localhost:7077?
> 
> 
> On Dec 21, 2014, at 9:28 AM, hlqv <hlqvuong@gmail.com> wrote:
> 
> Hi everyone!
> 
> In Mahout 1.0 Snapshot, I follow to this introduction
> https://mahout.apache.org/users/recommender/intro-cooccurrence-spark.html
> 
> and running the command below:
> 
> mahout spark-itemsimilarity --input hdfs://localhost:9000/test/logs.csv
> --output hdfs://localhost:9000/test/output/ --master spark://localhost:7077
> --filter1 purchase --filter2 view --itemIDColumn 2 --rowIDColumn 0
> --filterColumn 1
> 
> then I got the error:
> 
> Exception in thread "main" com.typesafe.config.ConfigException$Missing: No
> configuration setting found for key 'akka.event-handlers'
> at com.typesafe.config.impl.SimpleConfig.findKey(SimpleConfig.java:115)
> at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:136)
> at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:142)
> at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:150)
> at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:155)
> at com.typesafe.config.impl.SimpleConfig.getList(SimpleConfig.java:203)
> at
> 
> com.typesafe.config.impl.SimpleConfig.getHomogeneousUnwrappedList(SimpleConfig.java:260)
> at
> com.typesafe.config.impl.SimpleConfig.getStringList(SimpleConfig.java:318)
> at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:150)
> at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:470)
> at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
> at akka.actor.ActorSystem$.apply(ActorSystem.scala:104)
> at
> 
> org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
> at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:54)
> at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
> at
> 
> org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1446)
> at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
> at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1442)
> at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:56)
> at org.apache.spark.SparkEnv$.create(SparkEnv.scala:153)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:203)
> at
> 
> org.apache.mahout.sparkbindings.package$.mahoutSparkContext(package.scala:95)
> at
> 
> org.apache.mahout.drivers.MahoutSparkDriver.start(MahoutSparkDriver.scala:81)
> at
> 
> org.apache.mahout.drivers.ItemSimilarityDriver$.start(ItemSimilarityDriver.scala:128)
> at
> 
> org.apache.mahout.drivers.ItemSimilarityDriver$.process(ItemSimilarityDriver.scala:211)
> at
> 
> org.apache.mahout.drivers.ItemSimilarityDriver$$anonfun$main$1.apply(ItemSimilarityDriver.scala:116)
> at
> 
> org.apache.mahout.drivers.ItemSimilarityDriver$$anonfun$main$1.apply(ItemSimilarityDriver.scala:114)
> at scala.Option.map(Option.scala:145)
> at
> 
> org.apache.mahout.drivers.ItemSimilarityDriver$.main(ItemSimilarityDriver.scala:114)
> at
> 
> org.apache.mahout.drivers.ItemSimilarityDriver.main(ItemSimilarityDriver.scala)
> 
> I really don't know there is any mistake in my configuration, I installed
> Hadoop with CDH4 and Spark with the version for CDH4 and also put a sample
> file to HDFS.
> 
> Thanks for your help
> 
> 


Mime
View raw message