spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <>
Subject Re: Machine Learning on Spark [long rambling discussion email]
Date Wed, 24 Jul 2013 18:47:39 GMT
On Wed, Jul 24, 2013 at 1:46 AM, Nick Pentreath <>wrote:

> I also found Breeze to be very nice to work with and like the DSL - hence
> my question about why not use that? (Especially now that Breeze is actually
> just breeze-math and breeze-viz).

Matei addressed this from a higher level. I want to provide a little bit
more context. A common properties of a lot of high level Scala DSL
libraries is that simple operators tend to have high virtual function
overheads and also create a lot of temporary objects. And because the level
of abstraction is so high, it is fairly hard to debug / optimize

Reynold Xin, AMPLab, UC Berkeley

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message