spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@cs.berkeley.edu>
Subject Re: Needs a matrix library
Date Sat, 07 Sep 2013 00:09:08 GMT
They are asking about dedicated matrix libraries.

Neither GraphX nor Giraph are matrix libraries. These are systems that
handle large scale graph processing, which could possibly be modeled as
matrix computations.  Hama looks like a BSP framework, so I am not sure if
it has anything to do with matrix library either.

For very small matrices (3x3, 4x4), the cost of going through jni to do
native matrix operations will likely dominate the computation itself, so
you are probably better off with a simple unrolled for loop in Java.

I haven't looked into this myself, but I heard mahout-math is a decent
library.

--
Reynold Xin, AMPLab, UC Berkeley
http://rxin.org



On Sat, Sep 7, 2013 at 6:13 AM, Dmitriy Lyubimov <dlieu.7@gmail.com> wrote:

> keep forgetting this: what is graphx release roadmap?
>
> On Fri, Sep 6, 2013 at 3:04 PM, Konstantin Boudnik <cos@apache.org> wrote:
> > Would it be more logical to use GraphX ?
> >   https://amplab.cs.berkeley.edu/publication/graphx-grades/
> >
> > Cos
> >
> > On Fri, Sep 06, 2013 at 09:13PM, Mattmann, Chris A (398J) wrote:
> >> Thanks Roman, I was thinking Giraph too (knew it supported graphs but
> >> wasn't sure it supported matrices). If Giraph supports matrices, big +1.
> >>
> >> Cheers,
> >> Chris
> >>
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> Chris Mattmann, Ph.D.
> >> Senior Computer Scientist
> >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> Office: 171-266B, Mailstop: 171-246
> >> Email: chris.a.mattmann@nasa.gov
> >> WWW:  http://sunset.usc.edu/~mattmann/
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> Adjunct Assistant Professor, Computer Science Department
> >> University of Southern California, Los Angeles, CA 90089 USA
> >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >>
> >>
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Roman Shaposhnik <rvs@apache.org>
> >> Date: Friday, September 6, 2013 2:00 PM
> >> To: <dev@spark.incubator.apache.org>
> >> Cc: "dev@sis.apache.org" <dev@sis.apache.org>
> >> Subject: Re: Needs a matrix library
> >>
> >> >On Fri, Sep 6, 2013 at 1:33 PM, Mattmann, Chris A (398J)
> >> ><chris.a.mattmann@jpl.nasa.gov> wrote:
> >> >> Hey Martin,
> >> >>
> >> >> We may seriously consider using either Apache Hama here (which will
> >> >> bring in Hadoop):
> >> >
> >> >On that note I'd highly recommend taking a look at Apache Giraph
> >> >as well: http://giraph.apache.org/
> >> >
> >> >Thanks,
> >> >Roman.
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message