lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Jelsma <markus.jel...@openindex.io>
Subject RE: DocTransformer not always working
Date Wed, 21 Dec 2016 13:23:17 GMT
Yeah, that makes sense indeed.

Thanks!
Markus
 
-----Original message-----
> From:Chris Hostetter <hossman_lucene@fucit.org>
> Sent: Thursday 15th December 2016 19:44
> To: solr-user@lucene.apache.org
> Subject: RE: DocTransformer not always working
> 
> 
> : Well, i can work with this really fine knowing this, but does it make 
> : sense? I did assume (or be wrong in doing so) that fl=minhash:[binstr] 
> : should mean get that field and pass it through the transformer. At least 
> : i just now fell for it, maybe other shouldn't :)
> 
> that's what it *can* mean, but it's not -- fundementally -- what it means.
> 
> foo:[bar x=y ...] means run the "bar" transformer and request that it 
> uses the name "foo" as an output key in the resulting documents.
> 
> when "bar" is executing it knows what name it was asked to use, so it can 
> use that information for other purposes (like in your case: you can use 
> that as a stored field name to do some processing on) but there's no 
> reason "foo" has to be a real field name.
> 
> many processors don't treat the "name" special in any way, and in gneral a 
> processor should behave sanely if there is no name specified (ie: 
> "fl=[bar]" should be totally valid)
> 
> the key reason why it's not really a good idea to *force* the "name" used 
> in the response to match a "real" stored field is because it prevents you 
> from using multiple transformers on the same field, or from returning the 
> same field unmodified.
> 
> Another/Better way for you to have designed your transformer would have 
> been that the field to apply the binstr logic too should be specified as a 
> local param, ie...
> 
>   fl=minhash,b2_minhash:[binstr f=minhash base=2],b8_minhash:[binstr f=minhash base=16]
> 
> 
> ...see what i mean?
> 
> 
> 
> 
> : 
> : Anyway, thanks again today,
> : Markus
> : 
> : -----Original message-----
> : > From:Chris Hostetter <hossman_lucene@fucit.org>
> : > Sent: Wednesday 14th December 2016 23:14
> : > To: solr-user <solr-user@lucene.apache.org>
> : > Subject: Re: DocTransformer not always working
> : > 
> : > 
> : > Fairly certain you aren't overridding getExtraRequestFields, so when your 
> : > DocTransformer is evaluated it can'd find the field you want it to 
> : > transform.
> : > 
> : > By default, the ResponseWriters don't provide any fields that aren't 
> : > explicitly requested by the user, or specified as "extra" by the 
> : > DocTransformer.
> : > 
> : > IIUC you want the stored value of the "minhash" field to be available to 
> : > you, but the response writer code doesn't know that -- it just knows you 
> : > want "minhash" to be the output respons key for the "[binstr]" 
> : > transformer.
> : > 
> : > 
> : > Take a look at RawValueTransformerFactory as an example to borrow from.
> : > 
> : > 
> : > 
> : > 
> : > : Date: Wed, 14 Dec 2016 21:55:26 +0000
> : > : From: Markus Jelsma <markus.jelsma@openindex.io>
> : > : Reply-To: solr-user@lucene.apache.org
> : > : To: solr-user <solr-user@lucene.apache.org>
> : > : Subject: DocTransformer not always working
> : > : 
> : > : Hello - I just spotted an oddity with all two custom DocTransformers we sometimes
use on Solr 6.3.0. This particular transformer in the example just transforms a long (or int)
into a sequence of bits. I just use it as an convenience to compare minhashes with my eyeballs.
First example is very straightforward, fl=minhash:[binstr], show only the minhash field, but
as a bit sequence.
> : > : 
> : > : solr/search/select?omitHeader=true&wt=json&indent=true&rows=1&sort=id%20asc&q=*:*&fl=minhash:[binstr]
> : > : {
> : > :   "response":{"numFound":96933,"start":0,"docs":[
> : > :       {}]
> : > :   }}
> : > : 
> : > : The document is empty! This also happens with another transformer. The next
example i also request the lang field:
> : > : 
> : > : solr/search/select?omitHeader=true&wt=json&indent=true&rows=1&sort=id
asc&q=*:*&fl=lang,minhash:[binstr]
> : > : {
> : > :   "response":{"numFound":96933,"start":0,"docs":[
> : > :       {
> : > :         "lang":"nl"}]
> : > :   }}
> : > : 
> : > : Ok, at least i now get the lang field, but the transformed minhash is nowhere
to be seen. In the next example i request all fields and the transformed minhash:
> : > : 
> : > : /solr/search/select?omitHeader=true&wt=json&indent=true&rows=1&sort=id%20asc&q=*:*&fl=*,minhash:[binstr]
> : > : {
> : > :   "response":{"numFound":96933,"start":0,"docs":[
> : > :       {
> : > :         "minhash":"1110110101111010100011010010101111100001110110010111101101111010",
> : > :         ...other fields here
> : > :         "_version_":1553728923368423424}]
> : > :   }}
> : > : 
> : > : So it seems that right now, i can only use a transformer properly if i request
all fields. I believe it used to work with all three examples just as you would expect. But
since i haven't used transformers for a while, i don't know at which version it stopped working
like that (if it ever did of course :)
> : > : 
> : > : Did i mess something up or did a bug creep on me?
> : > : 
> : > : Thanks,
> : > : Markus
> : > : 
> : > 
> : > -Hoss
> : > http://www.lucidworks.com/
> : > 
> : 
> 
> -Hoss
> http://www.lucidworks.com/
> 

Mime
View raw message