spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Solve system of linear equations in Spark
Date Thu, 06 Oct 2016 08:42:31 GMT
Do you not just want to use linear regression?
https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala

Of course it requires a DataFrame-like input but that may be more natural
to begin with.

If the data set is small, then putting it on the driver and solving locally
with a library is pretty easy.

The Cholesky decomposition above doesn't solve the linear system itself,
but helps solve AtAx = Atb, because AtA and Atb are small and so that part
can be done locally.


On Thu, Oct 6, 2016 at 6:49 AM Cooper <ahmad.rabani.m@gmail.com> wrote:

> I have a system of linear equations in the form of Ax = b to solve in
> Spark.
>
> A is n by n
>
> b is n by 1
>
> I represent 'A' in the form of IndexedRowMatrix or RowMatrix and 'b' in the
> form of DenseMatrix or DenseVector.
>
> How can I solve this system to calculate the 'x' vector?
>
> If the suggested solution is  Cholesky Decomposition
> <
> https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/linalg/CholeskyDecomposition.scala
> >
> , would you please guide me through doing it as it is not part of the
> public
> API ? For example if the original matrix A is:
>
> 1,2,3,4
> 2,1,5,6
> 3,5,1,7
> 4,6,7,1
>
> and b is:
>
> 5,6,7,8
>
> What is passed as argument to the "solve" method ?
>
> Any other solution other than inversing 'A' would be very helpful.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Solve-system-of-linear-equations-in-Spark-tp27847.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>

Mime
View raw message