mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <>
Subject Re: Taste-GenericItemBasedRecommender
Date Sat, 12 Dec 2009 16:58:51 GMT
I've implemented this but it's still quite slow. Computing
recommendations goes from a couple hundred ms to 10 seconds. Nothing
wrong with this idea -- it's all the loading vectors and distributed
stuff that's weighing it down.

I think that's the culprit in fact, having to load all the column
vectors, since they're not light.

One approach is to make the user vectors more sparse by throwing out
data, though I don't like it so much.

One question -- in SparseVector, can't we internally remove entries
when they are set to 0.0? since implicitly missing entries are 0?

On Sat, Dec 5, 2009 at 10:27 PM, Ted Dunning <> wrote:
> The first approach requires access to all rows of M, but does the dot
> products using the sparse pattern of v.  The second approach inverts the
> nesting of the loops and moves the use of sparsity of v to the outer loop.
> This allows us to access only a small number of columns of M which is a huge
> win even if A is entirely in memory.  Secondarily, it also effectively lifts
> the highly repeated determination of the sparsity pattern of v out of the
> inner loop resulting in additional savings.

View raw message