lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Leir <rl...@leirtech.com>
Subject Re: Small Tokenization issue
Date Fri, 05 Jan 2018 08:32:00 GMT
Nawab

Look at classicTokenizer. It is a good choice if you have part numbers 
with hyphens. The second tokenizer on this page: 
https://lucene.apache.org/solr/guide/6_6/tokenizers.html

Cheers -- Rick


On 01/03/2018 04:52 PM, Shawn Heisey wrote:
> On 1/3/2018 1:56 PM, Nawab Zada Asad Iqbal wrote:
>> Thanks Emir, Erick.
>>
>> What i want to do is remove empty tokens after 
>> WordDelimiterGraphFilter ?
>> Is there any such option in WordDelimiterGraphFilter to not generate 
>> empty
>> tokens?
>
> I use LengthFilterFactory with a minimum of 1 and a maximum of 512 to 
> remove empty tokens.
>
> Thanks,
> Shawn
>


Mime
View raw message