spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Allman <...@allman.ms>
Subject Re: possible bug in Spark's ALS implementation...
Date Mon, 17 Mar 2014 23:41:59 GMT
You are correct, in the long run it doesn't matter which matrix you begin the
iterative process with. I was thinking in terms of doing a side-by-side
comparison to Oryx.

I've posted a bug report as SPARK-1262. I described the problem I found and
the mitigation strategy I've used. I think that this problem has many
possible solutions, so I'm omitting a patch to let the community hash out
the best approach. However, I will suggest we move to a pure Java
implementation of a linear system solver to provide better assurances of
correctness across platforms (differences in java.lang.Math notwithstanding)
and to make the implementation more transparent. It is not clear exactly
what native code JBlas is linked to and using for its solver. I suggested
the QR decomposition-based solvers provided by Colt and Commons Math as
candidate replacements.

Cheers.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/possible-bug-in-Spark-s-ALS-implementation-tp2567p2783.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message