lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: Boosting results
Date Mon, 10 Nov 2008 13:58:15 GMT
Michael McCandless wrote:
> But: it's slow to load a field for the first time.  LUCENE-1231 
> (column-stride fields) aims to greatly speed up the load time.
Test it out though. In some recent testing I was doing it was *way* 
faster than I thought it would be based on what I had been reading. Of 
course if every term is unique, its going to be worse, but even with 
like 10 mil docs and a few hundred thousand uniques, either I was doing 
something wrong, or even on my 4200rpm laptop hd, it loaded like nothing 
(of course even a second load and then a search is much slower than just 
a warmed search though). Was hoping to see some advantage with a payload 
implementation with LUCENE-831, but really didn't seem to...
> It's also memory-consuming.
> Finally, you might want to instead look at Solr, which provides facet 
> counting out of the box, rather than roll your own...
> Mike
> Stefan Trcek wrote:
>> On Friday 07 November 2008 18:46:17 Michael McCandless wrote:
>>> Sorting populates the field cache (internal to Lucene) for that
>>> field,   meaning it loads all values for all docs and holds them in
>>> memory. This makes the first query slow, and, consumes RAM, in
>>> proportion to how large your index is.
>> Can you direct me to the API how to access these cached values?
>> I'd like to have a function like: "List all unique values of the
>> categories (A, B, C...) for documents that match this query".
>> i.e. for a query "text:john" show up categories=(A,B)
>> Doc 1: category=A text=john
>> Doc 2: category=B text=mary
>> Doc 3: category=B text=john
>> Doc 4: category=C text=mary
>> This is intended for search refinement (I use about 200 categories).
>> Sorry for hijacking this thread.
>> Stefan
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message