lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Solrmails <solrma...@protonmail.com>
Subject Re: Heavy operations in PostFilter are heavy
Date Thu, 04 Jan 2018 07:50:25 GMT
Yes I do so. The Problem ist that the collect-Method is called for EVERY document the query
matches. Even if the User only wants to see like 10 documents. The Operation I have to perform
takes maybe 50ms/per document if have to process them singel. And maybe 30ms if I could get
a Document-List. But if the user e.g. uses an Wildcard query that matches maybe 100000 Documents
even 25ms are much to long.

I can't speedup my code anymore. Is there an other good place to do my checks? I tried to
remove the documents later but I don't know how to fetch more documents after removing them
on a later step. (Otherwise I would return maybe only 5 or zero documents even if the user
wants 10 and there are more documents available)

Sent with [ProtonMail](https://protonmail.com) Secure Email.

> -------- Original Message --------
> Subject: Re: Heavy operations in PostFilter are heavy
> Local Time: 3 January 2018 4:08 PM
> UTC Time: 3 January 2018 15:08
> From: arafalov@gmail.com
> To: solr-user <solr-user@lucene.apache.org>, Solrmails <solrmails@protonmail.com>
>
> Are you doing cache=false and cost > 100?
>
> See the recent article on the topic deep-dive, if you haven't:
> https://lucidworks.com/2017/11/27/caching-and-filters-and-post-filters/
>
> Regards,
> Alex.
>
> On 3 January 2018 at 05:31, Solrmails solrmails@protonmail.com wrote:
>
>> Hello,
>> I tried to write a Solr PostFilter to do filtering within the 'collect'-Method(DelegatingCollector).
I have to do some heavy operations within the 'collect'-Method. This isn't a problem for a
few results. But unfortunately it taks forever with 50 or more results. This is because I
have to do the checks for every single id again and can't process a list of ids within 'collect'.
>> Is there a better place to do PostFiltering? But I don't want to reimplement the
Solr Paging/Coursor-Feature to get my things to work.
>> Thank You
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message