mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johannes Schulte <johannes.schu...@gmail.com>
Subject Re: Implicit preferences
Date Mon, 11 Feb 2013 09:57:14 GMT
@Ken
Thanks for the hints...
I am coming from a payload based system so I am aware if them, however in
the lucene 3.6 branch boosting and payloads didn't work together (if you
set PayloadTermQuery.setIncludeSpanScore to false, they were ignored)

Besides that, there is no performance issue here so far so it's probably a
fine way to go, i was just curious...as for the IntField /  TrieIntField,
all the  range query / ordering benefits of it are overhead since the
integers just represent random indices into a vector. I might look into
indexing the integer bytes rather than the string separation...

@Ted
You are probably right with chosing 1 as term frequency, i forgot that the
most interesting information comes from the idf probably and using
cooccurrence counts as term frequency might make the combination with text
searches infeasible since the values lie in some totally different range.
Also I forgot that idf is per field so i might go for separating the hashed
values into their originating fields (search tern, item_id, category_id) .
This would still alllow to recombine them later when a use profile has to
be constructed.

I like to threshold with LLR.  That gives me a binary matrix.  Then I
> directly index that.
> The search engine provides very nice weights at this point.  I don't feel
> the need to adjust those weights because they have roughly the same form as
> learned weights are likely to have and because learning those weights would
> almost certainly result in over-fitting unless I go to quite a lot of
> trouble.
> Also, I have heard that at least one head-to-head test found that the
> native Solr term weighting actually out-performed several more intricate
> and explicit weighting schemes.  That can't be taken as evidence that
> Solr's weightings would perform better than whatever you have in mind, but
> it does provide interesting meta-evidence that the probability that a
> reasonably smart dev team is definitely not guaranteed to beat Solr's
> weighting by a large margin.  When you sit down to architect your system,
> you need to make decisions about where to spend your time and evidence like
> that is helpful to guess how much effort it would take to achieve different
> levels of performance.



I am also thresholding the counts with LLR. Every time i do this I take a
threshold of 10 since I loosely remember it  being about the 99% margin of
confidence in the chi square distribution. I got no clue however if anybody
wants something like 99% for recommendations or if 50% might be a better
value. What's your experience on that?

And do you apply a limit on the total number of docs per term, since there
could be big boolean queries tearing down the performance?

Thanks for all the input!



On Mon, Feb 11, 2013 at 7:20 AM, Ted Dunning <ted.dunning@gmail.com> wrote:

> On Sun, Feb 10, 2013 at 3:39 PM, Johannes Schulte <
> johannes.schulte@gmail.com> wrote:
>
> > ...
> > i am currently implementing a system of the same kind, LLR sparsified
> > "term"-cooccurrence vectors in lucene (since not a day goes by where i
> see
> > Ted praising this).
> >
>
> (turns red)
>
>
> > There are not only views and purchases, but also search terms, facets
> and a
> > lot more textual information to be included in the cooccurrence matrix
> (as
> > "input").
> > That's why i went with the feature hashing framework in mahout. This
> gives
> > small (hd/mem) user profiles and allows for reusing the vectors for click
> > prediction and/or clustering.
>
>
> This is a reasonable choice.  For recommendations, you might want to use
> direct encoding since it can be simpler to build a search index for
> recommending.
>
>
> > The main difference is that there's only two
> > fields in lucene with a lot of terms (Numbers), representing the
> features.
> > Two fields because i think predicting views (besides purchases) might in
> > some cases be better than predicting nothing.
> >
>
> OK.
>
>
> > I don't think it  should make a big differing in scoring because in a
> > vector space model used by most engines it's just, well a vector space
> and
> > i don't know if the field norm make sense after stripping values from the
> > term vectors with the LLR threshold.
> >
>
> Having separate fields is going to give separate total term counts.  That
> seems better to me, but I have to confess I have never rigorously tested
> that.
>
>
> > @Ted
> > > It is handy to simply use the binary values of the sparsified versions
> of
> > >these and let the search engine handle the weighting of different
> > >components at query time.
> >
> > Do you really want to omit the cooccurrence counts which would become the
> > term frequecies? how would the engine then weight different inputs
> against
> > each other?
> >
>
> I like to threshold with LLR.  That gives me a binary matrix.  Then I
> directly index that.
>
> The search engine provides very nice weights at this point.  I don't feel
> the need to adjust those weights because they have roughly the same form as
> learned weights are likely to have and because learning those weights would
> almost certainly result in over-fitting unless I go to quite a lot of
> trouble.
>
> Also, I have heard that at least one head-to-head test found that the
> native Solr term weighting actually out-performed several more intricate
> and explicit weighting schemes.  That can't be taken as evidence that
> Solr's weightings would perform better than whatever you have in mind, but
> it does provide interesting meta-evidence that the probability that a
> reasonably smart dev team is definitely not guaranteed to beat Solr's
> weighting by a large margin.  When you sit down to architect your system,
> you need to make decisions about where to spend your time and evidence like
> that is helpful to guess how much effort it would take to achieve different
> levels of performance.
>
> And, if anyone knows a
> > 1. smarter way to index the cooccurrence counts in lucene than a
> > tokenstream that emits a word k times for a cooccurrence count of k
> >
>
> You can use payloads or you can boost individual terms.
>
>
> > 2. way to avoid treating the (hashed) vector column indices as terms but
> > reusing them? It's a bit weird hashing to an int and then having the
> lucene
> > term dictionary treating them as string, mapping to another int
> >
>
> Why do we care about this?  These tokens get put onto documents that have
> additional data to help them make sense, but why do we care if the tokens
> look like numbers?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message