lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siegfried Goeschl <sgoes...@gmx.at>
Subject Re: Measuring QPS
Date Mon, 06 Apr 2015 18:41:31 GMT
Appreciated :-)

Siegfried Goeschl

> On 06 Apr 2015, at 20:31, Davis, Daniel (NIH/NLM) [C] <daniel.davis@nih.gov> wrote:
> 
> OK,
> 
> I have a lot of chutzpah posting that here ;)    The other guys answering the questions
can probably explain it better.
> I love showing off, however, so please forgive me.
> 
> -----Original Message-----
> From: Davis, Daniel (NIH/NLM) [C] 
> Sent: Monday, April 06, 2015 2:25 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Measuring QPS
> 
> Its very common to do autocomplete based on popular queries/titles over some sliding
time window.   Some enterprise search systems even apply age weighting so that they don't
need to re-index but continuously add to the index.   This way, they can do autocomplete based
on what's popular these days.
> 
> We use relevance/field boosts/phrase matching etc. to get the best guess about what results
they want to see.   This is similar - we use relevance, field boosting to guess what users
want to search for.   Zipf's law applies to searches as well as results.
> 
> -----Original Message-----
> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
> Sent: Monday, April 06, 2015 2:17 PM
> To: solr-user@lucene.apache.org
> Subject: Re: Measuring QPS
> 
> Hi Daniel,
> 
> interesting - I never thought of autocompletion but for keeping track of user behaviour
:-)
> 
> * the numbers are helpful for the online advertisement team to sell campaigns
> * it is used for sanity checks - sensible queries returning no results or returning too
many results
> 
> Cheers,
> 
> Siegfried Goeschl
> 
>> On 06 Apr 2015, at 20:04, Davis, Daniel (NIH/NLM) [C] <daniel.davis@nih.gov>
wrote:
>> 
>> Siegfried,
>> 
>> It is early days as yet.   I don't think we need a code drop.   AFAIK, none of our
current Solr applications autocomplete the search box based on popular query/title keywords.
  We have other applications that do that, but they don't use Solr.
>> 
>> Thanks again,
>> 
>> Dan
>> 
>> -----Original Message-----
>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>> Sent: Monday, April 06, 2015 1:42 PM
>> To: solr-user@lucene.apache.org
>> Subject: Re: Measuring QPS
>> 
>> Hi Dan,
>> 
>> at willhaben.at (customer of mine) two SOLR components were written 
>> for SOLR 3 and ported to SORL 4
>> 
>> 1) SlowQueryLog which dumps long-running search requests into a log 
>> file
>> 
>> 2) Most Frequent Search Terms allowing to query & filter the most 
>> frequent user search terms over the browser
>> 
>> Some notes along the line
>> 
>> 
>> * For both components I have the “GO" to open source them but I never 
>> had enough time to do that (shame on me) - see
>> https://issues.apache.org/jira/browse/SOLR-4056
>> 
>> * The Most Frequent Search Term component actually mimics a SOLR 
>> server you feed the user search terms so this might be a better 
>> solution in the long run. But this requires to have a separate SOLR 
>> core & ingest  plus GUI (check out SILK or ELK) - in other words more 
>> moving parts in production :-)
>> 
>> * If there is sufficient interest I can make a code drop on GitHub
>> 
>> Cheers,
>> 
>> Siegfried Goeschl
>> 
>> 
>> 
>>> On 06 Apr 2015, at 16:25, Davis, Daniel (NIH/NLM) [C] <daniel.davis@nih.gov>
wrote:
>>> 
>>> Siegfried,
>>> 
>>> This is a wonderful find.   The second presentation is a nice write-up of a large
number of free tools.   The first presentation prompts a question - did you add custom request
handlers/code to automate determination of best user search terms?   Did any of your custom
work end-up in Solr?
>>> 
>>> Thank you so much,
>>> 
>>> Dan
>>> 
>>> P.S. - your first presentation takes me back to seeing "Angrif der Klonkrieger"
in Berlin after a conference - Hayden Christensen was less annoying in German, because my
wife and I don't speak German ;)   I haven't thought of that in a while.
>>> 
>>> -----Original Message-----
>>> From: Siegfried Goeschl [mailto:sgoeschl@gmx.at]
>>> Sent: Saturday, April 04, 2015 4:54 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: Measuring QPS
>>> 
>>> Hi Dan,
>>> 
>>> I’m using JavaMelody for my SOLR production servers - gives you the 
>>> relevant HTTP stats (what’s happening now & historical data) plus JVM 
>>> monitoring as additional benefit. The servers are deployed on Tomcat 
>>> so I’m of little help regarding Jetty - having said that
>>> 
>>> * you need two Jars (javamelody & robin)
>>> * tinker with web.xml
>>> 
>>> Here are two of my presentations mentioning JavaMelody (plus some 
>>> other stuff)
>>> 
>>> http://people.apache.org/~sgoeschl/presentations/solr-from-developmen
>>> t
>>> -to-production-20121210.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/solr-from-developme
>>> n
>>> t-to-production-20121210.pdf>
>>> http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfor
>>> m
>>> ance-monitoring.pdf
>>> <http://people.apache.org/~sgoeschl/presentations/jsug-2015/jee-perfo
>>> r
>>> mance-monitoring.pdf>
>>> 
>>> Cheers,
>>> 
>>> Siegfried Goeschl
>>> 
>>>> On 03 Apr 2015, at 17:53, Shawn Heisey <apache@elyograg.org> wrote:
>>>> 
>>>> On 4/3/2015 9:37 AM, Davis, Daniel (NIH/NLM) [C] wrote:
>>>>> I wanted to gather QPS for our production Solr instances, but I was surprised
that the Admin UI did not contain this information.   We are running a mix of versions, but
mostly 4.10 at this point.   We are not using SolrCloud at present; that's part of why I'm
checking - I want to validate the size of our existing setup and what sort of SolrCloud setup
would be needed to centralize several of them.
>>>>> 
>>>>> What is the best way to gather QPS information?
>>>>> 
>>>>> What is the best way to add information like this to the Admin UI, if
I decide to take that step?
>>>> 
>>>> As of Solr 4.1 (three years ago), request rate information is 
>>>> available in the admin UI and via JMX.  In the admin UI, choose a 
>>>> core from the dropdown, click on Plugins/Stats, then QUERYHANDLER, 
>>>> and open the handler you wish to examine.  You have 
>>>> avgRequestsPerSecond, which is calculated for the entire runtime of 
>>>> the SolrCore, as well as 5minRateReqsPerSecond and 
>>>> 15minRateReqsPerSecond, which are far more useful pieces of information.
>>>> 
>>>> https://issues.apache.org/jira/browse/SOLR-1972
>>>> 
>>>> Thanks,
>>>> Shawn
>>>> 
>>> 
>> 
> 


Mime
View raw message