spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ulanov, Alexander" <>
Subject Which linear algebra interface to use within Spark MLlib?
Date Thu, 19 Mar 2015 03:09:17 GMT

Currently I am using Breeze within Spark MLlib for linear algebra. I would like to reuse previously
allocated matrices for storing the result of matrices multiplication, i.e. I need to use "gemm"
function C:=q*A*B+p*C, which is missing in Breeze (Breeze automatically allocates a new matrix
to store the result of multiplication). Also, I would like to minimize gemm calls that Breeze
does. Should I use mllib.linalg.BLAS functions instead? While it has gemm and axpy, it has
rather limited number of operations. For example, I need sum of the matrix by row or by columns,
or applying a function to all elements in a matrix. Also, MLlib Vector and Matrix interfaces
that linalg.BLAS operates seems to be rather undeveloped. Should I use plain netlib-java instead
(will it remain in MLlib in future releases)?

Best regards, Alexander

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message