lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charlie Hull <char...@flax.co.uk>
Subject Re: Debugging Solr Search results & Issues with Distributed IDF
Date Wed, 02 Jan 2019 11:53:02 GMT
On 01/01/2019 23:03, Lavanya Thirumalaisami wrote:
> 
> Hi,
> 
> I am trying to debug a query to find out why one documentgets more score than the other.
The below are two similar products.

You might take a look at OSC's Splainer http://splainer.io/ or some of 
the other tools I've written about recently at 
http://www.flax.co.uk/blog/2018/11/15/defining-relevance-engineering-part-4-tools/ 
- note that this also covers some commercial offerings (and also that 
I'm very happy to take any comments or additions!).

Cheers

Charlie
> 
> Below is the debug results I get from Solr admin console.
> 
>   "Doc1": "\n15.20965 = sum of:\n 4.7573533 = max of:\n    4.7573533= weight(All:2x
in 962) [], result of:\n           4.7573533 = score(doc=962,freq=2.0 =termFreq=2.0\n),
product of:\n       3.4598935 = idf(docFreq=1346, docCount=42836)\n        1.375
= tfNorm, computed from:\n          2.0 = termFreq=2.0\n          1.2 =
parameter k1\n          0.0 = parameter b (norms omitted forfield)\n  10.452296
= max of:\n    5.9166136 = weight(All:powerpoint in 962)[], result of:\n      5.9166136
=score(doc=962,freq=2.0 = termFreq=2.0\n), product of:\n        4.302992 = idf(docFreq=579,docCount=42836)\n       
1.375 = tfNorm,computed from:\n          2.0 =termFreq=2.0\n          1.2
= parameterk1\n          0.0 = parameter b (normsomitted for field)\n    10.452296
=weight(All:\"socket outlet\" in 962) [], result of:\n      10.452296 = score(doc=962,freq=2.0
=phraseFreq=2.0\n), product of:\n       7.60167 = idf(), sum of:\n         3.5370626
= idf(docFreq=1246, docCount=42836)\n          4.064607 = idf(docFreq=735,docCount=42836)\n       
1.375 = tfNorm,computed from:\n          2.0 =phraseFreq=2.0\n         
1.2 = parameterk1\n          0.0 = parameter b (normsomitted for field)\n",
> 
> "Doc15":"\n13.258003 = sum of:\n  5.7317085 = max of:\n    5.7317085 = weight(All:doubl
in 2122) [],result of:\n      5.7317085 =score(doc=2122,freq=2.0 = termFreq=2.0\n), product
of:\n        4.168515 = idf(docFreq=663,docCount=42874)\n        1.375 = tfNorm,computed
from:\n          2.0 =termFreq=2.0\n          1.2 = parameterk1\n         
0.0 = parameter b (normsomitted for field)\n    4.7657394 =weight(All:2x in 2122) [], result
of:\n     4.7657394 = score(doc=2122,freq=2.0 = termFreq=2.0\n), productof:\n       
3.4659925 =idf(docFreq=1339, docCount=42874)\n       1.375 = tfNorm, computed from:\n         2.0
= termFreq=2.0\n          1.2= parameter k1\n          0.0 = parameterb
(norms omitted for field)\n    5.390302= weight(All:2g in 2122) [], result of:\n     5.390302
= score(doc=2122,freq=2.0 = termFreq=2.0\n), product of:\n        3.9202197 = idf(docFreq=850,docCount=42874)\n       
1.375 = tfNorm,computed from:\n          2.0 = termFreq=2.0\n          1.2
= parameter k1\n          0.0 = parameter b (norms omitted forfield)\n  7.526294
= max of:\n    5.8597584 = weight(All:powerpoint in 2122)[], result of:\n      5.8597584
=score(doc=2122,freq=2.0 = termFreq=2.0\n), product of:\n        4.2616425 = idf(docFreq=604,docCount=42874)\n       
1.375 = tfNorm,computed from:\n          2.0 = termFreq=2.0\n          1.2
= parameter k1\n          0.0 = parameter b (norms omitted forfield)\n    7.526294
=weight(All:\"socket outlet\" in 2122) [], result of:\n      7.526294 = score(doc=2122,freq=1.0
=phraseFreq=1.0\n), product of:\n       7.526294 = idf(), sum of:\n         3.4955401
= idf(docFreq=1300, docCount=42874)\n          4.030754 = idf(docFreq=761,docCount=42874)\n       
1.0 = tfNorm,computed from:\n          1.0 =phraseFreq=1.0\n          1.2
= parameterk1\n          0.0 = parameter b (normsomitted for field)\n",
> 
>   
> 
> My Questions
> 
> 1.      IDF : I understand from solr documents that IDFis calculated for each separate
shards, I have added the following stats cacheconfig to solrconfig.xml and reloaded collection
> 
> <statsCacheclass="org.apache.solr.search.stats.ExactStatsCache"/>
> 
> But even after that there is no change incalculated IDF.
> 
> 2.      What are parameter b and parameter K1?
> 
> 3.      Why there are lots of parameters included in myDoc15 rather than Doc1?
> 
> Is there any documentations I can refer to understand thesolr query calculations in depth.
> 
> We are using  Solr 6.1in Cloud with 3 zookeepers and 3 masters and 3 replicas.
> 
> Regards,
> Lavanya
> 


-- 
Charlie Hull
Flax - Open Source Enterprise Search

tel/fax: +44 (0)8700 118334
mobile:  +44 (0)7767 825828
web: www.flax.co.uk
Mime
View raw message