spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiangrui Meng (JIRA)" <>
Subject [jira] [Resolved] (SPARK-1226) Import breeze to Spark mllib
Date Tue, 01 Apr 2014 05:34:16 GMT


Xiangrui Meng resolved SPARK-1226.

       Resolution: Fixed
    Fix Version/s: 1.0.0
         Assignee: Xiangrui Meng  (was: Kun Yang)

We use breeze to support underlying linear algebra operations in version 1.0.0.

> Import breeze to Spark mllib
> ----------------------------
>                 Key: SPARK-1226
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>            Reporter: Kun Yang
>            Assignee: Xiangrui Meng
>             Fix For: 1.0.0
> Breeze' math module includes many common linear algebra and optimization routines. It
is going to be very handy for developers if it is imported into mllib. 
> I am implementing ADMM (which is mentioned by the speaker in the Spark meetup) in mllib,
but there are few utility functions in current mllib and it is error-prone to code it from
> Many large scale machine learning algorithms need to solve several smaller scale algorithms
and combine the results (paralleled SGD) or to convert a disk resident problem into a memory
resident problem (linear regression). Breeze should be very useful in these cases.

This message was sent by Atlassian JIRA

View raw message