spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Pentreath (JIRA)" <>
Subject [jira] [Commented] (SPARK-15575) Remove breeze from dependencies?
Date Fri, 01 Jul 2016 11:38:11 GMT


Nick Pentreath commented on SPARK-15575:

Also related to the discussion on SPARK-15581 and here, replacing Breeze is not just about
replacing the core linalg operators. It also impacts L-BFGS and OWL-QN optimizers, which in
turn impacts linear/logistic regression and MLP. So we would need to have a plan to replace
that too.

cc [~avulanov] [~rxin] [~Xiangrui] [~josephkb] [~srowen] [~freiss] [~mboehm7] 

> Remove breeze from dependencies?
> --------------------------------
>                 Key: SPARK-15575
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: ML
>            Reporter: Joseph K. Bradley
> This JIRA is for discussing whether we should remove Breeze from the dependencies of
MLlib.  The main issues with Breeze are Scala 2.12 support and performance issues.
> There are a few paths:
> # Keep dependency.  This could be OK, especially if the Scala version issues are fixed
within Breeze.
> # Remove dependency
> ## Implement our own linear algebra operators as needed
> ## Design a way to build Spark using custom linalg libraries of the user's choice.  E.g.,
you could build MLlib using Breeze, or any other library supporting the required operations.
 This might require significant work.  See [SPARK-6442] for related discussion.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message