mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akshay Bhat <akshayub...@gmail.com>
Subject Re: possible alternative to very large scale SVD's
Date Sat, 09 Oct 2010 10:33:23 GMT
Interesting...
There is a much more recent work by their group [Faloutsos et al.] on mining
Peta scale graphs using Hadoop,
they have released their code called as Pegasus under Apache License.

http://www.cs.cmu.edu/~pegasus/

I am currently using their code, and It would be a worthwhile addition to
the Mahout.
I remember reading presentation by someone, (I guess it was by Jake Mannix)
about
how matrix multiplication is basis for most of the operations. Their group
seems to have
developed a block based algorithm to speed up multiplication of a Sparse
Matrix with a
dense column vector and using this multiplication as a primitive they have
implemented
 numerous algorithms optimized for use on hadoop/map-reduce.


On Fri, Oct 1, 2010 at 4:55 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

> Jake,
>
> You asked a bit ago about strategies for very large SVD's.
>
> I wonder if interpolative decompositions might be an avenue toward that.
>
> See, for instance, Less is More: Compact Matrix Decomposition for Large
> Sparse Graphs <http://www.cs.cmu.edu/~jimeng/papers/SunSDM07.pdf>
>
> The idea is that if your basis vectors are sparse, you might do much better
> in terms of space.
>



-- 
Akshay Uday Bhat.
Graduate Student, Computer Science, Cornell University
Website: http://www.akshaybhat.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message