lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christine Poerschke (BLOOMBERG/ LONDON)" <cpoersc...@bloomberg.net>
Subject Re: increasing number of threads for faceting in JSON format
Date Thu, 10 Dec 2020 17:44:47 GMT
Hello Arturas and Munendra!

In the "Currently, JSON facets have support for specifying the number of threads." sentence,
I wonder if perhaps a "does not" got inadvertently omitted i.e. "Currently, JSON facets does
not have support for specifying the number of threads." was intended?

Let me share what I learnt from digging into the code:

* "facet.threads" is for field value faceting [1] [2] but you're interested in (JSON) field
range faceting as well as JSON field value faceting.

* The area of the code [3] that does the JSON field range faceting shows no obvious threading
or parallelisation.

Hope that helps?

Regards,

Christine

[1] https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-parameters
[2] https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/request/SimpleFacets.java
[3] https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113

From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To:  solr-user@lucene.apache.org
Subject: Re: increasing number of threads for faceting in JSON format

Hi Munedra,

This is great that I can get things faster by reducing the gap and by
increasing the number of threads. How to reduce gaps I know: one can
replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I change
in the text below to increase the number of threads from one to 20?

Cheers,
Arturas

On Thu, Dec 3, 2020 at 1:54 PM Munendra S N <sn.munendra52@gmail.com> wrote:

> Hi,
>
> Currently, JSON facets have support for specifying the number of threads.
> In the above request, the range facet is computed over 2 years with a gap
> of 1 hour. By reducing the number of buckets, computation should become
> much faster
>
> Regards,
> Munendra S N
>
>
>
> On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika <mazeika@gmail.com> wrote:
>
> > Hi Solr-Users,
> >
> > I am trying to better understand the solr capabilities, how one can
> > formulate queries in JSON format as well as tweak parameters. Currently I
> > have a logs collection (ca 6GB large) with a dozen of attributes running
> in
> > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd start -h
> > localhost -p 5555 -m 4g)
> >
> > I am playing with faceting functionality in solr and query a couple of
> > attributes there. My typical query is:
> >
> > GET http://localhost:5555/solr/db/query
> > <http://arteika:5555/solr/logan/query> HTTP/1.1
> > content-type: application/json
> >
> > {
> >     "query"  : "*:*",
> >     "limit"  : 0,
> >     "facet": {
> >         "t" : {
> >             "type":  "terms",
> >             "field": "fcomp",
> >             "sort":  "index",
> >
> >             "facet": {
> >                 "t_buckets": {
> >                     "type":  "range",
> >                     "field": "t",
> >                     "sort": { "t": "asc" },
> >                     "start": "2018-05-02T17:00:00.000Z",
> >                     "end":   "2020-11-16T21:00:00.000Z",
> >                     "gap":   "+1HOUR"
> >                 }
> >             }
> >         },
> >     }
> > }
> >
> > not surprisingly, it takes a bit to compute the result, so I tried to
> > increase the number of threads. How do I do it in JSON format? I tried
> > adding
> >
> > {
> >     "params": {
> >         "facet.threads": 8
> >     },
> >     "query"  : "*:*",
> > ...
> > }
> >
> > and checked the jstack <pid> of the solr java process, but I still see
> only
> > one thread working.  Can I configure params through the params section?
> >
> > I also tried
> >
> > {
> >     "query"  : "*:*",
> >     "limit"  : 0,
> >     "facet": {
> >         "t" : {
> >             "type":  "terms",
> >             "field": "fcomp",
> >             "sort":  "index",
> >
> >             "facet": {
> >                 "t_buckets": {
> >                     "type":  "range",
> >                     "field": "t",
> >                     "sort": { "t": "asc" },
> >                     "start": "2018-05-02T17:00:00.000Z",
> >                     "end":   "2020-11-16T21:00:00.000Z",
> >                     "gap":   "+1HOUR"
> >                 }
> >             },
> >             "threads":8
> >         },
> >     }
> > }
> >
> > but this ran in one thread as well. Can I influence the number of threads
> > in the "facet" section of JSON?
> >
> > Cheers,
> > Arturas
> >
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message