lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Yano (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-11164) OriginalScoreFeature causes NullPointerException during feature logging with SolrCloud mode.
Date Mon, 14 Aug 2017 05:17:00 GMT

    [ https://issues.apache.org/jira/browse/SOLR-11164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16125251#comment-16125251
] 

Yuki Yano edited comment on SOLR-11164 at 8/14/17 5:16 AM:
-----------------------------------------------------------

[~cpoerschke] [~Jonathan GV] [~ryac]
I investigated these issues and found that {{score=0.0}} has special meaning for {{DocTransformer}}.

In SolrCloud, scores are calculated in the first-trip and score fields are filled after the
second-trip by the distributed node.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L1359

By this reason, in the second-trip, {{DocSlice}} is created with empty scores.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L356

While transforming documents, scores of {{DocSlice}} is referred by {{DocStreamer}} and set
{{0.0}} if scores are missing.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L170

Because {{DocSlice}} doesn't have scores, 0.0 is always passed to {{DocTransformer#transform}}
as the score. Without {{rq}} parameter, {{docsWereNotReranked=true}} and this causes the issue
reported in [SOLR-11180|https://issues.apache.org/jira/browse/SOLR-11180] and [SOLR-11220|https://issues.apache.org/jira/browse/SOLR-11220].

In conclusion, there are two problems with transforming {{OriginalScoreFeature}} in SolrCloud
mode.

1. Without {{rq}}, {{OriginalScoreFeature}} always returns 0.0 because the correct score isn't
passed to {{DocTransformer#transform}}.

2. With {{rq}}, {{FeatureTransformer}} causes NPE because the original query isn't set correctly
in {{LTRScoringQuery}}.


was (Author: yuyano):
[~cpoerschke] [~Jonathan GV] [~ryac]
I investigated these issues and found that {{score=0.0}} has special meaning for {{DocTransformer}}.

In SolrCloud, scores are calculated in the first-trip and score fields are filled after the
second-trip by the distributed node.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L1359

By this reason, in the second-trip, {{DocSlice}} is created with empty scores.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/core/src/java/org/apache/solr/handler/component/QueryComponent.java#L356

While transforming documents, scores of {{DocSlice}} is referred by {{DocStreamer}} and set
{{0.0}} if scores are missing.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/6.6.0/solr/core/src/java/org/apache/solr/response/DocsStreamer.java#L170

Because {{DocSlice}} doesn't have scores, 0.0 is always passed to {{DocTransformer#transform}}
as the score. Without {{rq}} parameter, {{docsWereNotReranked=true} and this causes the issue
reported in [SOLR-11180|https://issues.apache.org/jira/browse/SOLR-11180] and [SOLR-11220|https://issues.apache.org/jira/browse/SOLR-11220].

In conclusion, there are two problems with transforming {{OriginalScoreFeature}} in SolrCloud
mode.

1. Without {{rq}}, {{OriginalScoreFeature}} always returns 0.0 because the correct score isn't
passed to {{DocTransformer#transform}}.

2. With {{rq}}, {{FeatureTransformer}} causes NPE because the original query isn't set correctly
in {{LTRScoringQuery}}.

> OriginalScoreFeature causes NullPointerException during feature logging with SolrCloud
mode.
> --------------------------------------------------------------------------------------------
>
>                 Key: SOLR-11164
>                 URL: https://issues.apache.org/jira/browse/SOLR-11164
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: contrib - LTR
>    Affects Versions: 6.6
>            Reporter: Yuki Yano
>         Attachments: SOLR-11164.patch, SOLR-11164.patch
>
>
> In FeatureTransfer, OriginalScoreFeature uses original Query instance preserved in LTRScoringQuery
for the evaluation.
> This query is set in RankQuery#wrap during QueryComponent#process.
> With SolrCloud mode, document searches take two steps: finding top-N document ids, and
filling documents of found ids.
> In this case, FeatureTransformer works in the second step and tries to extract features
with LTRScoringQuery built in QueryComponent#prepare.
> However, because the second step doesn't call QueryComponent#process, the original query
of LTRScoringQuery remains null and this causes NullPointerException while evaluating OriginalScoreFeature.
> We can get the original query from ResultContext which is an argument of DocTransformer#setContext,
thus this problem can solve by using it if LTRScoringQuery doesn't have correct original query.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message