lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arturas Mazeika <maze...@gmail.com>
Subject Re: increasing number of threads for faceting in JSON format
Date Thu, 24 Dec 2020 14:19:28 GMT
Hi Christine,

Thanks a lot for the posts. Very impressive information (article as well as
the youtube video!)

Thanks a lot Merry Xmas and and Happy New Year!

Cheers,
Arturas

On Thu, Dec 24, 2020 at 11:03 AM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoerschke@bloomberg.net> wrote:

> Hello again Arturas.
>
> I meant to reply before but somehow lost track of it ... The "Lifecycle of
> a Solr Search Request" slides [1] and/or talk [2] may be of interest to you.
>
> Regards,
> Christine
>
> [1] https://home.apache.org/~hossman/rev2017/
> [2] https://youtu.be/qItRilJLj5o
>
> From: solr-user@lucene.apache.org At: 12/10/20 21:42:19To:
> solr-user@lucene.apache.org
> Subject: Re: increasing number of threads for faceting in JSON format
>
> Hi Christine Munendra et al,
>
> Wow, you dag into the code and checked weather threads are being blown in
> range and term queries! I wish one day to be able to do the same myself.
>
> How does one get to the level, so one can check the code herself? Is there
> like a nice primer or crash course, solr 101 so to say, things you did not
> learn in school about solr, but you wish you had learned web page? Well,
> I'll take this opportunity to scroll through the lines in the github. Your
> answer is very helpful.
>
> Cheers,
> Arturas
>
> On Thu, Dec 10, 2020 at 7:08 PM Munendra S N <sn.munendra52@gmail.com>
> wrote:
>
> > Thank you Christine.
> > Yeah, JSON facet does not support specifying threads.
> >
> >
> > On Thu, Dec 10, 2020, 11:15 PM Christine Poerschke (BLOOMBERG/ LONDON) <
> > cpoerschke@bloomberg.net> wrote:
> >
> > > Hello Arturas and Munendra!
> > >
> > > In the "Currently, JSON facets have support for specifying the number
> of
> > > threads." sentence, I wonder if perhaps a "does not" got inadvertently
> > > omitted i.e. "Currently, JSON facets does not have support for
> specifying
> > > the number of threads." was intended?
> > >
> > > Let me share what I learnt from digging into the code:
> > >
> > > * "facet.threads" is for field value faceting [1] [2] but you're
> > > interested in (JSON) field range faceting as well as JSON field value
> > > faceting.
> > >
> > > * The area of the code [3] that does the JSON field range faceting
> shows
> > > no obvious threading or parallelisation.
> > >
> > > Hope that helps?
> > >
> > > Regards,
> > >
> > > Christine
> > >
> > > [1]
> > >
> >
>
> https://lucene.apache.org/solr/guide/8_7/faceting.html#field-value-faceting-para
> meters
> > > [2]
> > >
> >
>
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/
> src/java/org/apache/solr/request/SimpleFacets.java
> <https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/request/SimpleFacets.java>
> > > [3]
> > >
> >
>
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/
> src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113
> <https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.7.0/solr/core/src/java/org/apache/solr/search/facet/FacetRangeProcessor.java#L112-L113>
> > >
> > > From: solr-user@lucene.apache.org At: 12/03/20 22:47:35To:
> > > solr-user@lucene.apache.org
> > > Subject: Re: increasing number of threads for faceting in JSON format
> > >
> > > Hi Munedra,
> > >
> > > This is great that I can get things faster by reducing the gap and by
> > > increasing the number of threads. How to reduce gaps I know: one can
> > > replace   "gap":   "+1HOUR" with   "gap":   "+1MONTH" What should I
> > change
> > > in the text below to increase the number of threads from one to 20?
> > >
> > > Cheers,
> > > Arturas
> > >
> > > On Thu, Dec 3, 2020 at 1:54 PM Munendra S N <sn.munendra52@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > Currently, JSON facets have support for specifying the number of
> > threads.
> > > > In the above request, the range facet is computed over 2 years with a
> > gap
> > > > of 1 hour. By reducing the number of buckets, computation should
> become
> > > > much faster
> > > >
> > > > Regards,
> > > > Munendra S N
> > > >
> > > >
> > > >
> > > > On Thu, Dec 3, 2020 at 1:52 PM Arturas Mazeika <mazeika@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Solr-Users,
> > > > >
> > > > > I am trying to better understand the solr capabilities, how one can
> > > > > formulate queries in JSON format as well as tweak parameters.
> > > Currently I
> > > > > have a logs collection (ca 6GB large) with a dozen of attributes
> > > running
> > > > in
> > > > > single server mode (F:\solr_deployment\solr-8.7.0\bin\solr.cmd
> start
> > -h
> > > > > localhost -p 5555 -m 4g)
> > > > >
> > > > > I am playing with faceting functionality in solr and query a couple
> > of
> > > > > attributes there. My typical query is:
> > > > >
> > > > > GET http://localhost:5555/solr/db/query
> > > > > <http://arteika:5555/solr/logan/query> HTTP/1.1
> > > > > content-type: application/json
> > > > >
> > > > > {
> > > > >     "query"  : "*:*",
> > > > >     "limit"  : 0,
> > > > >     "facet": {
> > > > >         "t" : {
> > > > >             "type":  "terms",
> > > > >             "field": "fcomp",
> > > > >             "sort":  "index",
> > > > >
> > > > >             "facet": {
> > > > >                 "t_buckets": {
> > > > >                     "type":  "range",
> > > > >                     "field": "t",
> > > > >                     "sort": { "t": "asc" },
> > > > >                     "start": "2018-05-02T17:00:00.000Z",
> > > > >                     "end":   "2020-11-16T21:00:00.000Z",
> > > > >                     "gap":   "+1HOUR"
> > > > >                 }
> > > > >             }
> > > > >         },
> > > > >     }
> > > > > }
> > > > >
> > > > > not surprisingly, it takes a bit to compute the result, so I tried
> to
> > > > > increase the number of threads. How do I do it in JSON format? I
> > tried
> > > > > adding
> > > > >
> > > > > {
> > > > >     "params": {
> > > > >         "facet.threads": 8
> > > > >     },
> > > > >     "query"  : "*:*",
> > > > > ...
> > > > > }
> > > > >
> > > > > and checked the jstack <pid> of the solr java process, but
I still
> > see
> > > > only
> > > > > one thread working.  Can I configure params through the params
> > section?
> > > > >
> > > > > I also tried
> > > > >
> > > > > {
> > > > >     "query"  : "*:*",
> > > > >     "limit"  : 0,
> > > > >     "facet": {
> > > > >         "t" : {
> > > > >             "type":  "terms",
> > > > >             "field": "fcomp",
> > > > >             "sort":  "index",
> > > > >
> > > > >             "facet": {
> > > > >                 "t_buckets": {
> > > > >                     "type":  "range",
> > > > >                     "field": "t",
> > > > >                     "sort": { "t": "asc" },
> > > > >                     "start": "2018-05-02T17:00:00.000Z",
> > > > >                     "end":   "2020-11-16T21:00:00.000Z",
> > > > >                     "gap":   "+1HOUR"
> > > > >                 }
> > > > >             },
> > > > >             "threads":8
> > > > >         },
> > > > >     }
> > > > > }
> > > > >
> > > > > but this ran in one thread as well. Can I influence the number of
> > > threads
> > > > > in the "facet" section of JSON?
> > > > >
> > > > > Cheers,
> > > > > Arturas
> > > > >
> > > >
> > >
> > >
> > >
> >
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message