lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandros Paramythis <alexandros.paramyt...@contexity.ch>
Subject Problems using a suggester component in the /select handler in cloud mode
Date Wed, 26 Jun 2019 08:48:13 GMT
Hi everyone,

Environment:

Solr 7.5.0, cloud mode (but problem should be identical in multiple 
versions, at least in 7.x)

Summary:

We have a Solr configuration that returns suggestions in the course of a 
normal search call (i.e., we have a 'suggest' component added to the 
'last-components' for '/select' request handler). This does not work in 
cloud mode, where we get an NPE in QueryComponent. This problem seems to 
have been reported in various forms in the past -- see for example [1] 
and [2] (links at the end of this email) -- but we couldn't find any 
resolution (or in-depth discussion for that matter).

In more detail:

We have a suggest component configured as follows:

   <searchComponent name="suggest" class="solr.SuggestComponent" >

         <lst name="suggester">
                 <str name="name">default</str>
                 <str 
name="classname">org.apache.solr.spelling.suggest.Suggester</str>
                 <str 
name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
                 <str name="storeDir">dict_default</str>
                 <str name="suggestAnalyzerFieldType">text_suggest</str>
                 <str name="field">text_suggest</str>
                 <str name="buildOnStartup">true</str>
                 <str name="buildOnCommit">true</str>
                 <str name="buildOnOptimize">true</str>
         </lst>

         <lst name="suggester">
                 <str name="name">suggest_phrase</str>
                 <str 
name="lookupImpl">org.apache.solr.spelling.suggest.fst.AnalyzingLookupFactory</str>
                 <str name="storeDir">dict_suggest_phrase</str>
                 <str 
name="suggestAnalyzerFieldType">text_suggest_phrase</str>
                 <str name="field">suggest_phrase</str>
                 <str name="buildOnStartup">true</str>
                 <str name="buildOnCommit">true</str>
                 <str name="buildOnOptimize">true</str>
         </lst>

         <lst name="suggester">
                 <str name="name">suggest_infix_shingle</str>
                 <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
                 <str name="indexPath">suggestInfixShingleDir</str>
                 <str 
name="suggestAnalyzerFieldType">text_suggest_phrase</str>
                 <str name="field">suggest_phrase</str>
                 <str name="buildOnOptimize">true</str>
                 <str name="buildOnCommit">true</str>
                 <str name="buildOnStartup">true</str>
         </lst>

         <lst name="suggester">
                 <str name="name">suggest_prefix</str>
                 <str name="classname">Suggester</str>
                 <str name="lookupImpl">AnalyzingLookupFactory</str>
                 <str 
name="suggestAnalyzerFieldType">text_suggest_prefix</str>
                 <str name="field">suggest_prefix</str>
                 <str name="buildOnOptimize">true</str>
                 <str name="buildOnCommit">true</str>
                 <str name="buildOnStartup">true</str>
         </lst>

   </searchComponent>


This component works without issued both in standalone and cloud mode, 
when used as the sole component in a handler, such as in the following 
excerpt:

         <requestHandler name="/suggest" class="solr.SearchHandler" 
startup="lazy">
                 <lst name="defaults">
                     <str name="suggest.dictionary">default</str>
                     <str name="suggest.dictionary">suggest_phrase</str>
                     <str 
name="suggest.dictionary">suggest_infix_shingle</str>
                     <str name="suggest.dictionary">suggest_prefix</str>
                     <str name="suggest">true</str>
                     <str name="suggest.count">10</str>
                     <str name="suggest.highlight">false</str>
                 </lst>
                 <arr name="components">
                         <str>suggest</str>
                 </arr>
         </requestHandler>


It also works when used along with other component in standalone mode, 
such as in the following excerpt, where we use the suggest component to 
get suggestions during a "normal" search call:

         <requestHandler name="/select" class="solr.SearchHandler">
                 <lst name="defaults">
                         <str name="echoParams">explicit</str>
                         <int name="rows">10</int>
                         <str name="df">text_search</str>

                         <str name="defType">edismax</str>

                         <str name="qf">title^5.0 subtitle^3.0

abstract^2.0 text_search</str>
                         <str name="pf">title^5.0 subtitle^3.0

abstract^2.0 text_search</str>
                         <str name="ps">4</str>
                         <str name="spellcheck">on</str>
                         <str name="spellcheck.dictionary">default</str>
                         <str name="spellcheck.extendedResults">true</str>
                         <str name="spellcheck.count">10</str>
                         <str name="spellcheck.alternativeTermCount">5</str>
                         <str name="spellcheck.maxResultsForSuggest">5</str>
                         <str name="spellcheck.collate">true</str>
                         <str 
name="spellcheck.collateExtendedResults">true</str>
                         <str name="spellcheck.maxCollationTries">10</str>
                         <str name="spellcheck.maxCollations">5</str>

                         <str name="suggest.dictionary">default</str>
                         <str name="suggest.dictionary">suggest_phrase</str>
                         <str 
name="suggest.dictionary">suggest_infix_shingle</str>
                         <str name="suggest.dictionary">suggest_prefix</str>
                         <str name="suggest">true</str>
                         <str name="suggest.count">10</str>
                         <str name="suggest.highlight">false</str>
                 </lst>

                 <arr name="last-components">
                         <str>suggest</str>
                         <str>spellcheck</str>
                 </arr>
         </requestHandler>

However, the above configuration does not work in cloud mode, where we 
get an NPE if a search call is made:

  o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
	at org.apache.solr.handler.component.QueryComponent.unmarshalSortValues(QueryComponent.java:1034)
	at org.apache.solr.handler.component.QueryComponent.mergeIds(QueryComponent.java:885)
	at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:585)
	at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:564)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:515)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:531)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
	at java.lang.Thread.run(Thread.java:748)


As far as we can tell, this is due to the fact that the failing code in 
QueryComponent assumes that the shard response always contains a 
'sort_values' section:

         NamedList sortFieldValues = 
(NamedList)(srsp.getSolrResponse().getResponse().get("sort_values"));
         NamedList unmarshalledSortFieldValues = unmarshalSortValues(ss, 
sortFieldValues, schema);

(the exception occurs in unmarshalSortValues(), which assumes the 
sortFieldValues parameter is not null)

Are we doing something wrong that is preventing 'sort_values' from being 
present in the shard response? Or is their absence OK, but the code 
fails to account for it? Our guess is the second, because 
umarshalSortValues() does check if the values are empty or not, and 
handles that.

Can anyone help us with this issue? Thanks in advance for any pointers!

Alex

[1] 
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201711.mbox/%3CCACKWnqOv=HFQwTxd7SwEqSTj-im+43FE+c9_zbTCPMWsp8vL_w@mail.gmail.com%3E

[2] https://issues.apache.org/jira/browse/SOLR-12060


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message