spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@cs.berkeley.edu>
Subject Re: Machine Learning on Spark [long rambling discussion email]
Date Wed, 24 Jul 2013 18:47:39 GMT
On Wed, Jul 24, 2013 at 1:46 AM, Nick Pentreath <nick.pentreath@gmail.com>wrote:

>
> I also found Breeze to be very nice to work with and like the DSL - hence
> my question about why not use that? (Especially now that Breeze is actually
> just breeze-math and breeze-viz).
>


Matei addressed this from a higher level. I want to provide a little bit
more context. A common properties of a lot of high level Scala DSL
libraries is that simple operators tend to have high virtual function
overheads and also create a lot of temporary objects. And because the level
of abstraction is so high, it is fairly hard to debug / optimize
performance.




--
Reynold Xin, AMPLab, UC Berkeley
http://rxin.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message