mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Mannix <jake.man...@gmail.com>
Subject Re: Taste-GenericItemBasedRecommender
Date Thu, 03 Dec 2009 18:55:27 GMT
I guess yes, it depends on what form your data is in.  If you never have a
need
to produce the transpose, then this is costly, sure.  If you are already
building your
matrix out of something like say, a lucene index, then you already *have*
the
transpose.   Or if you have another use for doing a MR job to produce the
transpose and then keeping it around for other purposes, I guess.

But yeah, in the case at hand, maybe that transposition is a bunch of work.
But is
it more work to do my two MR jobs than to do your two MR jobs?  Both look
like two
full passes over the data, right?

  -jake

On Thu, Dec 3, 2009 at 10:46 AM, Ted Dunning <ted.dunning@gmail.com> wrote:

> I have always tried to avoid transposing the data matrix.
>
> During my design phase for this, I was working on patch-wise patterns.
> About then, Chris Dyer tried the simpler approach for a machine translation
> problem and got very good results.
>
> The major problem with the transpose is that it requires a MR job of its
> own
> that is nearly as expensive as the multiply.   The combiner really makes a
> huge difference to the inner product approach.
>
> On Thu, Dec 3, 2009 at 10:31 AM, Jake Mannix <jake.mannix@gmail.com>
> wrote:
>
> > Wait, this is just doing A'A?  Am I misunderstanding, or is this not most
> > easily done by first transposing A into A', and then doing the outer
> > products instead of the inner products
> >
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message