lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: IntField to IntPoint
Date Fri, 07 Jun 2019 14:45:17 GMT
Omitting norms and the like only matters for text fields, primitives (numerics, boolean, string)
don’t have any  of that information.

You really have no choice but to re-index to jump from 4->7. Or, I should say you’re
completely unsupported  and you will have to deal with any anomalies. I suppose if the only
thing you care about is non-textual data you might be OK, but it's iffy at best.

And you’ll have to play low-level games with Lucene to rewrite the segments with points
rather than ints.

Good  luck!
Erick

I’ll wager that you’ll be faster to re-index, painful though it may be rather than write
custom code  to do this.

> On Jun 7, 2019, at 1:40 AM, Riccardo Tasso <riccardo.tasso@gmail.com> wrote:
> 
> Thanks Erik for your answer.
> 
> Unfortunately I should migrate the index for time reasons. Maybe in a
> second moment we will have the opportunity to reindex.
> 
> Our use case is to classify documents in the index with lucene queries,
> hence we're not really interested in ranking or sorting (which could be
> relevant for the "norms case"). Do you think that migrating and reindexing
> only the numeric fields could compromise the results returned by any query
> (term, boolean, range, phrase, prefix)?
> 
> Il giorno mer 5 giu 2019 alle ore 17:41 Erick Erickson <
> erickerickson@gmail.com> ha scritto:
> 
>> You cannot upgrade more than one major version, you must re-index from
>> scratch. There’s a long discussion of why, but basically it’s summed up by
>> this quote from Robert Muir:
>> 
>> “I think the key issue here is Lucene is an index not a database. Because
>> it is a lossy index and does not retain all of the user's data, its not
>> possible to safely migrate some things automagically. In the norms case
>> IndexWriter needs to re-analyze the text ("re-index") and compute stats to
>> get back the value, so it can be re-encoded. The function is y = f(x) and
>> if x is not available its not possible, so lucene can't do it.”
>> 
>> This has always been true, before 8x it would just  fail silently as  you
>> have found. Solr/Lucene starts up but don’t  work quite as expected. As of
>> Lucene 8x, Lucene (and therefore Solr) will not even open an index that
>> has  _ever_ been touched by Lucene 6x, no matter what intervening steps
>> have been taken. Or in general,  Lucene/Solr X will  not  open indexes
>> touched by X-2, starting with 8x rather than behave unexpectedly.
>> 
>> Best,
>> Erick
>> 
>>> On Jun 5, 2019, at 8:27 AM, Riccardo Tasso <riccardo.tasso@gmail.com>
>> wrote:
>>> 
>>> Hello everybody,
>>> I have a (very big) lucene 4 index with documents using IntField. On that
>>> field, which should be stored and sortable, I should search and execute
>>> range queries.
>>> 
>>> I've tried to upgrade it from 4 to 7 with IndexUpgrader but I observed
>> that
>>> IntFields aren't searchable anymore.
>>> 
>>> Which is the most efficient way to convert IntFields to IntPoints, which
>>> are stored and sortable?
>>> 
>>> Thanks,
>>> Riccardo
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message