spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Bradley <jos...@databricks.com>
Subject Re: Is there any Sparse Matrix implementation in Spark/MLib?
Date Sun, 01 Mar 2015 00:25:39 GMT
Hi Shahab,

There are actually a few distributed Matrix types which support sparse
representations: RowMatrix, IndexedRowMatrix, and CoordinateMatrix.
The documentation has a bit more info about the various uses:
http://spark.apache.org/docs/latest/mllib-data-types.html#distributed-matrix

The Spark 1.3 RC includes a new one: BlockMatrix.

But since these are distributed, they are represented using RDDs, so they
of course will not be as fast as computations on smaller, locally stored
matrices.

Joseph

On Fri, Feb 27, 2015 at 4:39 AM, Ritesh Kumar Singh <
riteshoneinamillion@gmail.com> wrote:

> try using breeze (scala linear algebra library)
>
> On Fri, Feb 27, 2015 at 5:56 PM, shahab <shahab.mokari@gmail.com> wrote:
>
>> Thanks a lot Vijay, let me see how it performs.
>>
>> Best
>> Shahab
>>
>>
>> On Friday, February 27, 2015, Vijay Saraswat <vijay@saraswat.org> wrote:
>>
>>> Available in GML --
>>>
>>> http://x10-lang.org/x10-community/applications/global-
>>> matrix-library.html
>>>
>>> We are exploring how to make it available within Spark. Any ideas would
>>> be much appreciated.
>>>
>>> On 2/27/15 7:01 AM, shahab wrote:
>>>
>>>> Hi,
>>>>
>>>> I just wonder if there is any Sparse Matrix implementation available
>>>> in Spark, so it can be used in spark application?
>>>>
>>>> best,
>>>> /Shahab
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: user-help@spark.apache.org
>>>
>>>
>

Mime
View raw message