spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reza Zadeh <r...@databricks.com>
Subject Re: sparse x sparse matrix multiplication
Date Thu, 06 Nov 2014 22:50:11 GMT
See this thread for examples of sparse matrix x sparse matrix:
https://groups.google.com/forum/#!topic/spark-users/CGfEafqiTsA

We thought about providing matrix multiplies on CoordinateMatrix, however,
the matrices have to be very dense for the overhead of having many little
(i, j, value) objects to be worth it. For this reason, we are focused on
doing block matrix multiplication first. The goal is version 1.3.

Best,
Reza

On Wed, Nov 5, 2014 at 11:48 PM, Wei Tan <wtan@us.ibm.com> wrote:

> I think Xiangrui's ALS code implement certain aspect of it. You may want
> to check it out.
> Best regards,
> Wei
>
> ---------------------------------
> Wei Tan, PhD
> Research Staff Member
> IBM T. J. Watson Research Center
>
>
> [image: Inactive hide details for Xiangrui Meng ---11/05/2014 01:13:40
> PM---You can use breeze for local sparse-sparse matrix multiplic]Xiangrui
> Meng ---11/05/2014 01:13:40 PM---You can use breeze for local sparse-sparse
> matrix multiplication and then define an RDD of sub-matri
>
> From: Xiangrui Meng <mengxr@gmail.com>
> To: Duy Huynh <duy.huynh.uiv@gmail.com>
> Cc: user <user@spark.incubator.apache.org>
> Date: 11/05/2014 01:13 PM
> Subject: Re: sparse x sparse matrix multiplication
> ------------------------------
>
>
>
> You can use breeze for local sparse-sparse matrix multiplication and
> then define an RDD of sub-matrices
>
> RDD[(Int, Int, CSCMatrix[Double])] (blockRowId, blockColId, sub-matrix)
>
> and then use join and aggregateByKey to implement this feature, which
> is the same as in MapReduce.
>
> -Xiangrui
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
>

Mime
View raw message