spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiangrui Meng <men...@gmail.com>
Subject Re: MLlib Collaborative Filtering failed to run with rank 1000
Date Fri, 03 Oct 2014 23:07:58 GMT
It would be really helpful if you can help test the scalability of the
new ALS impl: https://github.com/mengxr/spark-als/blob/master/src/main/scala/org/apache/spark/ml/SimpleALS.scala
. It should be faster and more scalable, but the code is messy now.

Best,
Xiangrui

On Fri, Oct 3, 2014 at 11:57 AM, jw.cmu <jinliangwei1@gmail.com> wrote:
> Thanks, Xiangrui.
>
> I didn't check the test error yet. I agree that rank 1000 might overfit for
> this particular dataset. Currently I'm just running some scalability tests -
> I'm trying to see how large the model can be scaled to given a fixed amount
> of hardware.
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MLlib-Collaborative-Filtering-failed-to-run-with-rank-1000-tp15692p15697.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message