lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Solr7 org.apache.lucene.index.IndexUpgrader
Date Fri, 24 Nov 2017 14:24:58 GMT
On 11/23/2017 11:31 PM, Leo Prince wrote:
> We were using bit older version Solr 4.10.2 and upgrading to Solr7.
> 
> We have like 4mil records in one of the core which is of course pretty
> huge, hence re-sourcing the index is nearly impossible and re-querying from
> source Solr to Solr7 is also going to be an exhausting effort.

I hate to burst your bubble here ... but 4 million docs is pretty small 
for a Solr index.  I have one index that's a hundred times larger, and 
there are people with *billions* of documents in SolrCloud.

> Hence, I tried to upgrade the Index using
> org.apache.lucene.index.IndexUpgrader.
<snip>
> IndexUpgrader ran just fine without any errors. but got this error with
> initializing the core.
> 
> *java.lang.IllegalStateException:java.lang.IllegalStateException:
> unexpected docvalues type NONE for field '_version_' (expected=NUMERIC).
> Re-index with correct docvalues type.*
> Being said, I am using Classic Schema and used default managed-schema file
> as classic schema.xml.

This error means that the existing index didn't have docValues on the 
_version_ field, but the new version does.  At some point in 6.x, a 
whole bunch of field classes were changed to have docValues by default. 
You'll need to explicitly add 'docValues="false"' to the field 
definition to use an older index with a newer version.  But based on 
some things you said later, this may be the least of the problems you're 
running into.

> When comparing schema of 4.10.2 with that of 7.1.0, I see the field type
> names have changed like follows
> 
> *<fieldType name="pint" class="solr.IntPointField"
> docValues="true"/><fieldType name="pfloat" class="solr.FloatPointField"
> docValues="true"/><fieldType name="plong" class="solr.LongPointField"
> docValues="true"/><fieldType name="pdouble" class="solr.DoublePointField"
> docValues="true"/>*
> 
> Earlier until Solr6, it was int, float, long and double (*with out P at the
> beginning*). I read in docs, old field type names are deprecated in Solr7
> and have to use everything starting with "*P*" which enhances the
> performances. Hence in this context,
> 
> 1, The error I got
> *java.lang.IllegalStateException:java.lang.IllegalStateException,
> *Is it because my index data synced and upgraded contains old field type
> names and new Solr7 schema contains new field type names..? Being
> said, my IndexUpgrade
> completed without any errors.

You *cannot* change the classes being used for your fields (which the 
fieldType changes you have described will do) on an existing index and 
expect Solr to work.  If you change the class on a field, you must 
eliminate the current index and reindex from scratch.

> 2, How to sort out the error in 1, if my assessment correct.? Since my data
> is too large such that it's hard to re-source or re-query, is there any
> other work arounds to migrate the index if IndexUpgrade is not an option to
> upgrade index to 7.

You would need to keep the schema the same for the upgrade, except that 
you would need to disable docValues on some of your fields to get rid of 
the error you encountered.  You won't be able to take advantage of some 
of the new capability in the new version unless you re-engineer your 
config/schema and reindex.

Upgrading an index, especially through three major versions, is 
generally not recommended.  I always reindex when upgrading Solr, 
especially to a new major version, because Solr evolves quickly.

Thanks,
Shawn

Mime
View raw message