lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Petersen, Robert" <rober...@buy.com>
Subject RE: parsing debug output for readability
Date Thu, 10 Jan 2013 23:16:42 GMT
Hi Erik,

Thanks, debug.explain.structured=true helps a lot!  Could you also tell me what these `#8;#0;#0;#0;#1;
strings represent in the debug output?  Are they some internal representation of the field
name/value combos in the query?  They come out like this:  
fieldWeight(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1; in 122108)
tf(termFreq(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;)=1)

...and this:
<str name="description">
boost(+(mfgPartNo:canon title:canon^1.1 titleSort:canon^1.1 taxonomyCategoryName:canon^20.0
moreWords:canon) +(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^105.0 taxonomyCategoryTypeId:`#8;#0;#0;#0;#0;)
+(ConstantScore(boosted:[1 TO *]^1000.0)^1000.0 boosted:`#8;#0;#0;#0;#0;) +(storeId:`#8;#0;#0;#0;#1;^80.0
storeId:`#8;#0;#0;#0;#2;^5.0 storeId:`#8;#0;#0;#0;#3; storeId:`#8;#0;#0;#0;#4; storeId:`#8;#0;#0;#0;#5;^5.0
storeId:`#8;#0;#0;#0;#6; storeId:`#8;#0;#0;#0;#7;^80.0 storeId:`#8;#0;#0;#0;#8;^80.0 ConstantScore(storeId:[9
TO *]^60.0)^60.0) +(ConstantScore(sales:[10 TO *]^200.0)^200.0 ConstantScore(sales:[5 TO 9]^190.0)^190.0
ConstantScore(sales:[1 TO 4]^180.0)^180.0 sales:`#8;#0;#0;#0;#0;) +(ConstantScore(views:[51
TO *]^250.0)^250.0 ConstantScore(views:[10 TO 50]^30.0)^30.0 ConstantScore(views:[1 TO 9]^10.0)^10.0
views:`#8;#0;#0;#0;#0;) +(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^75.0


Thanks again,
R


-----Original Message-----
From: Erik Hatcher [mailto:erik.hatcher@gmail.com] 
Sent: Thursday, January 10, 2013 12:49 PM
To: solr-user@lucene.apache.org
Subject: Re: parsing debug output for readability

Robert -

Two options here:

  - Use debug.explain.structured <http://wiki.apache.org/solr/CommonQueryParameters#debug.explain.structured>

  - Use wt=ruby&indent=on and it'll come out in an indented fashion browser friendly manner,
but even in XML it should come out with whitespace and newlines in the actual XML source (browsers
render it ugly though)

	Erik


On Jan 10, 2013, at 15:35 , Petersen, Robert wrote:

> Hi Solr Users,
> 
> Can someone give me some good parsing rules of thumb to make the debug explain output
human readable?  I found this cool site for visualizing the output but our queries are too
complex and break their parser:  http://explain.solr.pl
> 
> I tried adding new lines plus indenting after every 'sum of:' and 'product of:', adding
new lines between every number = <something>, and adding new lines plus un-indenting
after every comma, but that doesn't quite seem right.  Thanks for any input.
> 
> Here is one of our explains:
> <lst name="explain">
> <str name="243030948">
> 46.044563 = (MATCH) boost(+(mfgPartNo:canon title:canon^1.1 titleSort:canon^1.1 taxonomyCategoryName:canon^20.0
moreWords:canon) +(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^105.0 taxonomyCategoryTypeId:`#8;#0;#0;#0;#0;)
+(ConstantScore(boosted:[1 TO *]^1000.0)^1000.0 boosted:`#8;#0;#0;#0;#0;) +(storeId:`#8;#0;#0;#0;#1;^80.0
storeId:`#8;#0;#0;#0;#2;^5.0 storeId:`#8;#0;#0;#0;#3; storeId:`#8;#0;#0;#0;#4; storeId:`#8;#0;#0;#0;#5;^5.0
storeId:`#8;#0;#0;#0;#6; storeId:`#8;#0;#0;#0;#7;^80.0 storeId:`#8;#0;#0;#0;#8;^80.0 ConstantScore(storeId:[9
TO *]^60.0)^60.0) +(ConstantScore(sales:[10 TO *]^200.0)^200.0 ConstantScore(sales:[5 TO 9]^190.0)^190.0
ConstantScore(sales:[1 TO 4]^180.0)^180.0 sales:`#8;#0;#0;#0;#0;) +(ConstantScore(views:[51
TO *]^250.0)^250.0 ConstantScore(views:[10 TO 50]^30.0)^30.0 ConstantScore(views:[1 TO 9]^10.0)^10.0
views:`#8;#0;#0;#0;#0;) +(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^75.0 taxonomyCategoryTypeId:`#8;#0;#0;#0;#0;),pow(sum(log(sum(product(int(boosted),const(9000.0)),product(product(int(image),int(stocked)),const(300.0)),product(product(int(image),int(taxonomyCategoryTypeId)),const(300.0)),product(product(int(image),int(sales)),const(150.0)),product(int(stocked),const(2.0)),product(int(sales),const(2.0)),int(views))),const(1.0)),const(3.0))),
product of: 0.36104107 = (MATCH) sum of: 0.026820535 = (MATCH) product of: 0.04470089 = (MATCH)
sum of: 0.03307638 = (MATCH) weight(mfgPartNo:canon in 122108), product of: 0.0059631695 =
queryWeight(mfgPartNo:canon), product of: 11.093556 = idf(docFreq=672, maxDocs=16277616) 5.3753454E-4
= queryNorm 5.546778 = (MATCH) fieldWeight(mfgPartNo:canon in 122108), product of: 1.0 = tf(termFreq(mfgPartNo:canon)=1)
11.093556 = idf(docFreq=672, maxDocs=16277616) 0.5 = fieldNorm(field=mfgPartNo, doc=122108)
0.0057238983 = (MATCH) weight(title:canon^1.1 in 122108), product of: 0.00424859 = queryWeight(title:canon^1.1),
product of: 1.1 = boost 7.1853147 = idf(docFreq=33522, maxDocs=16277616) 5.3753454E-4 = queryNorm
1.3472465 = (MATCH) fieldWeight(title:canon in 122108), product of: 1.0 = tf(termFreq(title:canon)=1)
7.1853147 = idf(docFreq=33522, maxDocs=16277616) 0.1875 = fieldNorm(field=title, doc=122108)
0.005900612 = (MATCH) weight(moreWords:canon in 122108), product of: 0.0038275106 = queryWeight(moreWords:canon),
product of: 7.1204925 = idf(docFreq=35767, maxDocs=16277616) 5.3753454E-4 = queryNorm 1.5416318
= (MATCH) fieldWeight(moreWords:canon in 122108), product of: 1.7320508 = tf(termFreq(moreWords:canon)=3)
7.1204925 = idf(docFreq=35767, maxDocs=16277616) 0.125 = fieldNorm(field=moreWords, doc=122108)
0.6 = coord(3/5) 0.058011983 = (MATCH) product of: 0.116023965 = (MATCH) sum of: 0.116023965
= (MATCH) weight(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^105.0 in 122108), product of: 0.08092295
= queryWeight(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^105.0), product of: 105.0 = boost 1.4337585
= idf(docFreq=10549013, maxDocs=16277616) 5.3753454E-4 = queryNorm 1.4337585 = (MATCH) fieldWeight(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;
in 122108), product of: 1.0 = tf(termFreq(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;)=1) 1.4337585
= idf(docFreq=10549013, maxDocs=16277616) 1.0 = fieldNorm(field=taxonomyCategoryTypeId, doc=122108)
0.5 = coord(1/2) 2.6876872E-4 = (MATCH) product of: 5.3753745E-4 = (MATCH) sum of: 5.3753745E-4
= (MATCH) weight(boosted:`#8;#0;#0;#0;#0; in 122108), product of: 5.37536E-4 = queryWeight(boosted:`#8;#0;#0;#0;#0;),
product of: 1.0000027 = idf(docFreq=16277571, maxDocs=16277616) 5.3753454E-4 = queryNorm 1.0000027
= (MATCH) fieldWeight(boosted:`#8;#0;#0;#0;#0; in 122108), product of: 1.0 = tf(termFreq(boosted:`#8;#0;#0;#0;#0;)=1)
1.0000027 = idf(docFreq=16277571, maxDocs=16277616) 1.0 = fieldNorm(field=boosted, doc=122108)
0.5 = coord(1/2) 0.17403002 = (MATCH) product of: 0.7831351 = (MATCH) sum of: 0.75088304 =
(MATCH) weight(storeId:`#8;#0;#0;#0;#8;^80.0 in 122108), product of: 0.17969431 = queryWeight(storeId:`#8;#0;#0;#0;#8;^80.0),
product of: 80.0 = boost 4.178669 = idf(docFreq=677816, maxDocs=16277616) 5.3753454E-4 = queryNorm
4.178669 = (MATCH) fieldWeight(storeId:`#8;#0;#0;#0;#8; in 122108), product of: 1.0 = tf(termFreq(storeId:`#8;#0;#0;#0;#8;)=1)
4.178669 = idf(docFreq=677816, maxDocs=16277616) 1.0 = fieldNorm(field=storeId, doc=122108)
0.032252073 = (MATCH) ConstantScore(storeId:[9 TO *]^60.0)^60.0, product of: 60.0 = boost
5.3753454E-4 = queryNorm 0.22222222 = coord(2/9) 0.026876727 = (MATCH) product of: 0.10750691
= (MATCH) sum of: 0.10750691 = (MATCH) ConstantScore(sales:[10 TO *]^200.0)^200.0, product
of: 200.0 = boost 5.3753454E-4 = queryNorm 0.25 = coord(1/4) 0.03359591 = (MATCH) product
of: 0.13438363 = (MATCH) sum of: 0.13438363 = (MATCH) ConstantScore(views:[51 TO *]^250.0)^250.0,
product of: 250.0 = boost 5.3753454E-4 = queryNorm 0.25 = coord(1/4) 0.041437127 = (MATCH)
product of: 0.08287425 = (MATCH) sum of: 0.08287425 = (MATCH) weight(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^75.0
in 122108), product of: 0.057802103 = queryWeight(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;^75.0),
product of: 75.0 = boost 1.4337585 = idf(docFreq=10549013, maxDocs=16277616) 5.3753454E-4
= queryNorm 1.4337585 = (MATCH) fieldWeight(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1; in 122108),
product of: 1.0 = tf(termFreq(taxonomyCategoryTypeId:`#8;#0;#0;#0;#1;)=1) 1.4337585 = idf(docFreq=10549013,
maxDocs=16277616) 1.0 = fieldNorm(field=taxonomyCategoryTypeId, doc=122108) 0.5 = coord(1/2)
127.53276 = pow(sum(log(sum(product(int(boosted)=0,const(9000.0)),product(product(int(image)=1,int(stocked)=2),const(300.0)),product(product(int(image)=1,int(taxonomyCategoryTypeId)=1),const(300.0)),product(product(int(image)=1,int(sales)=64),const(150.0)),product(int(stocked)=2,const(2.0)),product(int(sales)=64,const(2.0)),int(views)=171)),const(1.0)),const(3.0))
> </str>
> 
> 
> 




Mime
View raw message