lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexis Aravena Silva <aarav...@itsofteg.com>
Subject Problems creating index for suggestions
Date Tue, 04 Apr 2017 21:11:51 GMT
Hi,


I'm creating an index for suggestions, when I rebuild the index with 8 documents, Solr creates
a temp file that consumes over 20GB in the process and It takes more than 10 minutes in reindex,
what is the problem?, It's illogic that Solr takes so long and consumes such size of my disk:



Filed Type Definition:


<fieldType name="text_suggestion" class="solr.TextField" positionIncrementGap="100" multiValued="true">
      <analyzer type="index">
        <tokenizer class="solr.LowerCaseTokenizerFactory"/>
        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"
/>
        <filter class="solr.EdgeNGramFilterFactory" minGramSize="1" maxGramSize="15" />
        <filter class="solr.ASCIIFoldingFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.LowerCaseTokenizerFactory"/>
         <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"
/>
        <filter class="solr.ASCIIFoldingFilterFactory"/>
      </analyzer>
    </fieldType>


Suggester Configuration:


<searchComponent name="suggest" class="solr.SuggestComponent">
    <lst name="suggester">
      <str name="name">fuzzySuggester</str>
      <str name="lookupImpl">FuzzyLookupFactory</str>
      <str name="indexPath">fuzzy_suggestions</str>
      <str name="dictionaryImpl">DocumentDictionaryFactory</str>
      <str name="field">_sugerencia_</str>
      <str name="payloadField">idTipoRegistro</str>
      <str name="suggestAnalyzerFieldType">text_suggestion</str>
      <str name="buildOnStartup">false</str>
      <str name="buildOnCommit">true</str>
    </lst>
    <lst name="suggester">
      <str name="name">infixSuggester</str>
      <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
      <str name="indexPath">infix_suggestions</str>
      <str name="dictionaryImpl">DocumentDictionaryFactory</str>
      <str name="field">_sugerencia_</str>
      <str name="payloadField">idTipoRegistro</str>
      <str name="suggestAnalyzerFieldType">text_suggestion</str>
      <str name="buildOnStartup">false</str>
      <str name="buildOnCommit">true</str>
    </lst>
  </searchComponent>
  <requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy" >
    <lst name="defaults">
      <str name="suggest">true</str>
      <str name="suggest.dictionary">infixSuggester</str>
      <str name="suggest.dictionary">fuzzySuggester</str>
      <str name="suggest.onlyMorePopular">true</str>
      <str name="suggest.count">10</str>
      <str name="suggest.collate">true</str>
    </lst>
    <arr name="components">
      <str>suggest</str>
    </arr>
  </requestHandler>



I rebuild the suggestions once by week, that's why I set buildOnCommit = true.


Regards.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message