spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shahab <shahab.mok...@gmail.com>
Subject Re: Is there any Sparse Matrix implementation in Spark/MLib?
Date Sun, 01 Mar 2015 10:53:29 GMT
Thanks Josef for the comments, I think I need to do some benchmarking.

best,
/Shahab

On Sun, Mar 1, 2015 at 1:25 AM, Joseph Bradley <joseph@databricks.com>
wrote:

> Hi Shahab,
>
> There are actually a few distributed Matrix types which support sparse
> representations: RowMatrix, IndexedRowMatrix, and CoordinateMatrix.
> The documentation has a bit more info about the various uses:
> http://spark.apache.org/docs/latest/mllib-data-types.html#distributed-matrix
>
> The Spark 1.3 RC includes a new one: BlockMatrix.
>
> But since these are distributed, they are represented using RDDs, so they
> of course will not be as fast as computations on smaller, locally stored
> matrices.
>
> Joseph
>
> On Fri, Feb 27, 2015 at 4:39 AM, Ritesh Kumar Singh <
> riteshoneinamillion@gmail.com> wrote:
>
>> try using breeze (scala linear algebra library)
>>
>> On Fri, Feb 27, 2015 at 5:56 PM, shahab <shahab.mokari@gmail.com> wrote:
>>
>>> Thanks a lot Vijay, let me see how it performs.
>>>
>>> Best
>>> Shahab
>>>
>>>
>>> On Friday, February 27, 2015, Vijay Saraswat <vijay@saraswat.org> wrote:
>>>
>>>> Available in GML --
>>>>
>>>> http://x10-lang.org/x10-community/applications/global-
>>>> matrix-library.html
>>>>
>>>> We are exploring how to make it available within Spark. Any ideas would
>>>> be much appreciated.
>>>>
>>>> On 2/27/15 7:01 AM, shahab wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I just wonder if there is any Sparse Matrix implementation available
>>>>> in Spark, so it can be used in spark application?
>>>>>
>>>>> best,
>>>>> /Shahab
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>>>> For additional commands, e-mail: user-help@spark.apache.org
>>>>
>>>>
>>
>

Mime
View raw message