spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shivaram Venkataraman <shiva...@eecs.berkeley.edu>
Subject Re: Solving linear equations
Date Thu, 23 Oct 2014 17:13:06 GMT
We have been working on large scale QR decompositions which would fit
this problem well -- TSQR
[https://github.com/amplab/ml-matrix/blob/master/src/main/scala/edu/berkeley/cs/amplab/mlmatrix/TSQR.scala]
for instance can be used to solve the least squares system if you have
more equations than variables (over-determined)

We plan to merge some of these into Spark as a part of
https://issues.apache.org/jira/browse/SPARK-3434 as well

Thanks
Shivaram

On Thu, Oct 23, 2014 at 1:54 AM, Sean Owen <sowen@cloudera.com> wrote:
> The 0 vector is a trivial solution. Is the data big, such that it
> can't be computed on one machine? if so I assume this system is
> over-determined. You can use a decomposition to find a least-squares
> solution, but the SVD is overkill and in any event distributed
> decompositions don't exist in the project. You can solve it a linear
> regression as Mr Das says.
>
> If it's small enough to fit locally you should just use a matrix
> library to solve Ax = b with the QR decomposition or something, with
> Breeze or Commons Math or octave or R. Lots of options if it's
> smallish.
>
> On Thu, Oct 23, 2014 at 12:15 AM, Martin Enzinger
> <martin.enzinger@gmail.com> wrote:
>> Hi,
>>
>> I'm wondering how to use Mllib for solving equation systems following this
>> pattern
>>
>> 2*x1 + x2 + 3*x3 + .... + xn = 0
>> x1 + 0*x2 + 3*x3 + .... + xn = 0
>> ..........
>> ..........
>> 0*x1 + x2 + 0*x3 + .... + xn = 0
>>
>> I definitely still have some reading to do to really understand the direct
>> solving techniques, but at the current state of "knowledge" SVD could help
>> me with this right?
>>
>> Can you point me to an example or a tutorial?
>>
>> best regards
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message