lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clemens Wyss DEV <clemens...@mysign.ch>
Subject AW: Occasionally getting error in solr suggester component.
Date Tue, 02 Jun 2015 14:04:47 GMT
Thanks!

-----Urspr√ľngliche Nachricht-----
Von: Erick Erickson [mailto:erickerickson@gmail.com] 
Gesendet: Montag, 1. Juni 2015 17:26
An: solr-user@lucene.apache.org
Betreff: Re: Occasionally getting error in solr suggester component.

Attach suggester.build=true or suggester.buildAll=true to any request to suggester to rebuild.
OR
add
buildOnStartup or buildOnCommit or buildOnOptimize to the definition in solrConfig.

BUT:
building can be a _very_ expensive operation. For document-based indexes, the build process
reads through _all_ of the _stored_ documents in your index, and that can take many minutes
so I recommend against these options for a large index, and strongly recommend you test these
with a large corpus.

Best,
Erick



On Mon, Jun 1, 2015 at 4:01 AM, Clemens Wyss DEV <clemensdev@mysign.ch> wrote:
> Lucene 5.1:
> I am (also) facing
> "java.lang.IllegalStateException: suggester was not built"
>
> At the very moment no new documents seem tob e added to the index/core. Will a reboot
"sanitize" the index/core?
>
> I (still) have
> <str> name="buildOnCommit">true</str>
>
> How can I tell Solr to peridoically update the suggestions? If not possible per configuration
(in solrconfig.xml), what ist he preferred approach through SolrJ?
>
> Thx
> Clemens
>
> -----Urspr√ľngliche Nachricht-----
> Von: Michael Sokolov [mailto:msokolov@safaribooksonline.com]
> Gesendet: Donnerstag, 15. Januar 2015 19:52
> An: solr-user@lucene.apache.org
> Betreff: Re: Occasionally getting error in solr suggester component.
>
> That sounds like a good approach to me.  Of course it depends how often you commit, and
what your tolerance is for delay in having suggestions appear, but it sounds as if you have
a good understanding of the tradeoffs there.
>
> -Mike
>
> On 1/15/15 10:31 AM, Dhanesh Radhakrishnan wrote:
>> Hi,
>>  From Solr 4.7 onwards, the implementation of this Suggester is 
>> changed. The old SpellChecker based search component is replaced with 
>> a new suggester that utilizes Lucene suggester module. The latest 
>> Solr download is preconfigured with this new suggester I;m using Solr 
>> 4.10 and suggestion are based on query  /suggest instead of /spell.
>> So what I did is that in changed to <str 
>> name="buildOnCommit">false</str> Its not good that each time rebuild 
>> the index on  commit , however, I would like to build the index on 
>> certain time period, say 1 hour.
>> The lookup data will be built only when requested by URL parameter 
>> suggest.build=true
>>
>> "http://localhost:8983/solr/ha/suggest?suggest.build=true"
>>
>> So this will rebuild the index again and the changes will reflect in 
>> the suggester.
>>
>> There are certain pros and cons for this.
>> Issue is that the change will reflect only on certain time interval, 
>> here 1 hour. Advantage is that we can avoid the  rebuilt index  on 
>> every commit or optimize.
>>
>> Is this the right way ?? or any that I missed ???
>>
>> Regards
>> dhanesh s.r
>>
>>
>>
>>
>> On Thu, Jan 15, 2015 at 3:20 AM, Michael Sokolov < 
>> msokolov@safaribooksonline.com> wrote:
>>
>>> did you build the spellcheck index using spellcheck.build as 
>>> described
>>> here: https://cwiki.apache.org/confluence/display/solr/Spell+Checking ?
>>>
>>> -Mike
>>>
>>>
>>> On 01/14/2015 07:19 AM, Dhanesh Radhakrishnan wrote:
>>>
>>>> Hi,
>>>> Thanks for the reply.
>>>> As you mentioned in the previous mail I changed buildOnCommit=false 
>>>> in solrConfig.
>>>> After that change, suggestions are not working.
>>>> In Solr 4.7 introduced a new approach based on a dedicated 
>>>> SuggestComponent I'm using that component to build suggestions and 
>>>> lookup implementation is "AnalyzingInfixLookupFactory"
>>>> Is there any work around ??
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Jan 14, 2015 at 12:47 AM, Michael Sokolov < 
>>>> msokolov@safaribooksonline.com> wrote:
>>>>
>>>>   I think you are probably getting bitten by one of the issues 
>>>> addressed in
>>>>> LUCENE-5889
>>>>>
>>>>> I would recommend against using buildOnCommit=true - with a large 
>>>>> index this can be a performance-killer.  Instead, build the index 
>>>>> yourself using the Solr spellchecker support
>>>>> (spellcheck.build=true)
>>>>>
>>>>> -Mike
>>>>>
>>>>>
>>>>> On 01/13/2015 10:41 AM, Dhanesh Radhakrishnan wrote:
>>>>>
>>>>>   Hi all,
>>>>>> I am experiencing a problem in Solr SuggestComponent Occasionally

>>>>>> solr suggester component throws an  error like
>>>>>>
>>>>>> Solr failed:
>>>>>> {"responseHeader":{"status":500,"QTime":1},"error":{"msg":"sugges
>>>>>> t
>>>>>> er
>>>>>> was
>>>>>> not built","trace":"java.lang.IllegalStateException: suggester 
>>>>>> was not built\n\tat 
>>>>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>>>>>> lookup(AnalyzingInfixSuggester.java:368)\n\tat
>>>>>> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.
>>>>>> lookup(AnalyzingInfixSuggester.java:342)\n\tat
>>>>>> org.apache.lucene.search.suggest.Lookup.lookup(Lookup.java:240)\n
>>>>>> \ tat org.apache.solr.spelling.suggest.SolrSuggester.
>>>>>> getSuggestions(SolrSuggester.java:199)\n\tat
>>>>>> org.apache.solr.handler.component.SuggestComponent.
>>>>>> process(SuggestComponent.java:234)\n\tat
>>>>>> org.apache.solr.handler.component.SearchHandler.handleRequestBody
>>>>>> (
>>>>>> SearchHandler.java:218)\n\tat
>>>>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(
>>>>>> RequestHandlerBase.java:135)\n\tat
>>>>>> org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.
>>>>>> handleRequest(RequestHandlers.java:246)\n\tat
>>>>>> org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
>>>>>> org.apache.solr.servlet.SolrDispatchFilter.execute(
>>>>>> SolrDispatchFilter.java:777)\n\tat
>>>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>>>>>> SolrDispatchFilter.java:418)\n\tat
>>>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>>>>>> SolrDispatchFilter.java:207)\n\tat
>>>>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(
>>>>>> ApplicationFilterChain.java:243)\n\tat
>>>>>> org.apache.catalina.core.ApplicationFilterChain.doFilter(
>>>>>> ApplicationFilterChain.java:210)\n\tat
>>>>>> org.apache.catalina.core.StandardWrapperValve.invoke(
>>>>>> StandardWrapperValve.java:225)\n\tat
>>>>>> org.apache.catalina.core.StandardContextValve.invoke(
>>>>>> StandardContextValve.java:123)\n\tat
>>>>>> org.apache.catalina.core.StandardHostValve.invoke(
>>>>>> StandardHostValve.java:168)\n\tat 
>>>>>> org.apache.catalina.valves.ErrorReportValve.invoke(
>>>>>> ErrorReportValve.java:98)\n\tat
>>>>>> org.apache.catalina.valves.AccessLogValve.invoke(
>>>>>> AccessLogValve.java:927)\n\tat
>>>>>> org.apache.catalina.valves.RemoteIpValve.invoke(
>>>>>> RemoteIpValve.java:680)\n\tat
>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(
>>>>>> StandardEngineValve.java:118)\n\tat
>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(
>>>>>> CoyoteAdapter.java:407)\n\tat
>>>>>> org.apache.coyote.http11.AbstractHttp11Processor.process(
>>>>>> AbstractHttp11Processor.java:1002)\n\tat
>>>>>> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.
>>>>>> process(AbstractProtocol.java:579)\n\tat
>>>>>> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.
>>>>>> run(JIoEndpoint.java:312)\n\tat
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(
>>>>>> ThreadPoolExecutor.java:1145)\n\tat
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>>>>> ThreadPoolExecutor.java:615)\n\tat
>>>>>> java.lang.Thread.run(Thread.java:745)\n","code":500}}
>>>>>>
>>>>>> This is not freequently happening, but idexing and suggestor 
>>>>>> component working togethere  this error will occur.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> In solr config
>>>>>>
>>>>>> <searchComponent name="suggest" class="solr.SuggestComponent">
>>>>>>        <lst name="suggester">
>>>>>>          <str name="name">haSuggester</str>
>>>>>>          <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
>>>>>> <!--
>>>>>> org.apache.solr.spelling.suggest.fst -->
>>>>>>          <str name="suggestAnalyzerFieldType">textSpell</str>
>>>>>>          <str name="dictionaryImpl">DocumentDictionaryFactory</str>
>>>>>>    <!--
>>>>>> org.apache.solr.spelling.suggest.HighFrequencyDictionaryFactory -->
>>>>>>          <str name="field">name</str>
>>>>>>          <str name="weightField">packageWeight</str>
>>>>>>          <str name="buildOnCommit">true</str>
>>>>>>        </lst>
>>>>>>      </searchComponent>
>>>>>>
>>>>>>      <requestHandler name="/suggest" class="solr.SearchHandler"
>>>>>> startup="lazy">
>>>>>>        <lst name="defaults">
>>>>>>          <str name="suggest">true</str>
>>>>>>          <str name="suggest.count">10</str>
>>>>>>        </lst>
>>>>>>        <arr name="components">
>>>>>>          <str>suggest</str>
>>>>>>        </arr>
>>>>>>      </requestHandler>
>>>>>>
>>>>>> Can any one suggest where to look to figure out this error and 
>>>>>> why these errors are occurring?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> dhanesh s.r
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>>
>>>>>>
>>
>
Mime
View raw message