mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pat Ferrel <...@occamsmachete.com>
Subject Re: mahout spark-itemsimilarity does not work on EMR 4.3
Date Wed, 24 Feb 2016 17:16:41 GMT
Another way to get Mahout item-similarity based recommender is to use the Universal Recommender
here: https://github.com/actionml/template-scala-parallel-universal-recommendation/tree/v0.3.0

It includes an event input pipeline, periodic Mahout+Spark based model generation and a realtime
serving component. It can be set up to use EMR for the Spark jobs. BTW by removing the Redis
part recs can be made on realtime data and queries returned in realtime. PM me if you want
to know more.

On Feb 24, 2016, at 12:25 AM, Zhun Shen <shenzhunallen@gmail.com> wrote:

My use case:
I use mahout to do my recommender, most of our application build on Spark, I use Spark to
prepare the dataset and then I want use mahout on Spark to calculate items similarity, 
then Spark will be use to format the output and save the result to Redis. After that, frontend
can get the recommend data from Redis.

You can see,  I just want to do ETL job, recommend training and deploy jobs all in Spark stack
in AWS EMR.

By the way, when will next release of EMR ? 


> On Feb 24, 2016, at 11:24 AM, Jonathan Kelly <jonathakamzn@gmail.com> wrote:
> 
> The next release of EMR will include Mahout 0.11.1 so that Mahout on Spark
> works with Spark 1.6. Sorry for any inconvenience until then.
> 
> By the way, I'm interested to know what your use case is for running Mahout
> on Spark, so please feel free to PM me if you are able to share any details.
> 
> Thank you,
> Jonathan
> On Tue, Feb 23, 2016 at 6:06 PM Andrew Palumbo <ap.dev@outlook.com> wrote:
> 
>> Please update to Mahout 0.11.1 for spark versions > 1.3.
>> 
>> -------- Original message --------
>> From: Zhun Shen <shenzhunallen@gmail.com>
>> Date: 02/23/2016 8:57 PM (GMT-05:00)
>> To: user@mahout.apache.org
>> Subject: mahout spark-itemsimilarity does not work on EMR 4.3
>> 
>> Hi,
>> mahout version: 0.11.0
>> EMR version: 4.3
>> spark version: 1.6.0
>> 
>> I try to run mahout spark-itemsimilarity on AWS EMR, but it told me that
>> “MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
>> Cannot find Spark classpath. Is 'SPARK_HOME' set?”, Is it a bug for EMR or
>> I use mahout spark in a wrong way ?
>> 



Mime
View raw message