spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiangrui Meng <men...@gmail.com>
Subject Re: mllib.recommendation Design
Date Tue, 17 Feb 2015 23:19:57 GMT
The current ALS implementation allow pluggable solvers for
NormalEquation, where we put CholeskeySolver and NNLS solver. Please
check the current implementation and let us know how your constraint
solver would fit. For a general matrix factorization package, let's
make a JIRA and move our discussion there. -Xiangrui

On Fri, Feb 13, 2015 at 7:46 AM, Debasish Das <debasish.das83@gmail.com> wrote:
> Hi,
>
> I am bit confused on the mllib design in the master. I thought that core
> algorithms will stay in mllib and ml will define the pipelines over the
> core algorithm but looks like in master ALS is moved from mllib to ml...
>
> I am refactoring my PR to a factorization package and I want to build it on
> top of ml.recommendation.ALS (possibly extend from ml.recommendation.ALS
> since first version will use very similar RDD handling as ALS and a
> proximal solver that's being added to breeze)
>
> https://issues.apache.org/jira/browse/SPARK-2426
> https://github.com/scalanlp/breeze/pull/321
>
> Basically I am not sure if we should merge it with recommendation.ALS since
> this is more generic than recommendation. I am considering calling it
> ConstrainedALS where user can specify different constraint for user and
> product factors (Similar to GraphLab CF structure).
>
> I am also working on ConstrainedALM where the underlying algorithm is no
> longer ALS but nonlinear alternating minimization with constraints.
> https://github.com/scalanlp/breeze/pull/364
> This will let us do large rank matrix completion where there is no need to
> construct gram matrices. I will open up the JIRA soon after getting initial
> results
>
> I am bit confused that where should I add the factorization package. It
> will use the current ALS test-cases and I have to construct more test-cases
> for sparse coding and PLSA formulations.
>
> Thanks.
> Deb

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message