lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Khludnev <>
Subject Re: Solr large boolean filter
Date Sun, 16 Jun 2013 09:43:13 GMT
FieldCacheTermsFilter is an option. You need to create own QParserPlugin
which yields FieldCacheTermsFilter, hook him as ..&fq={!idsqp
Mind disabling caching! Mind term ecoding due to field type!

I also suggest to check how much it spend for tokenization. Once a day I've
got some profit by using efficient encoding for this param (try fixed
length or vint)

There is a one more gain when the core query is highly selective and id
filter is weakly selective, in  this case using explicit PostFiltering
(what a hack btw) is desired. see

>From my experience the proper solution for such problems is moving to one
of the joins or ExternalFileField.

On Sun, Jun 16, 2013 at 2:49 AM, Igor Kustov <> wrote:

> I know i'm not the first one with this problem.
> I'm currently using solr 4.2.1 with approximately 10 mln documents in the
> index.
> The index is updated frequently.
> The filter_query is just a one big boolean or query by id.
> fq=id:(1 2 3 4 ... 50950)
> ids list is always different and not sequential.
> The problem is that query performance not so well, as you can imagine.
> In some particular cases i'm able to do filtering based on different
> fields,
> but in some cases (like 30-40% of all queries) i'm still end up with this
> large id filter.
> I'm looking for the ways to improve this query performance.
> It doesn't seem like solr join could be applied there.
> Another option that I found is to somehow use Lucene FieldCacheTermsFilter.
> Does it worth a try?
> Maybe i've missed some other options?
> --
> View this message in context:
> Sent from the Solr - User mailing list archive at

Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message