lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leo Prince <leoprince.discussi...@gmail.com>
Subject Re: Solr7 org.apache.lucene.index.IndexUpgrader
Date Mon, 27 Nov 2017 09:58:14 GMT
Hi Daniel,

Thanks for the help.

Actually I have two major cores. One I have primary document store as MySQL
and I can populate and re-index data from MySQL. However the other core
with 40mil, is keeping as primary store (with stored=true), I get the fact
that it's not a good practice, however due to some reason, I don't have any
duplicate data store for this core. It's a low priority dynamic data piling
up directly to Solr. In this context, is there a better method other than
keeping the current Solr (older) as data source..? Any other work
arounds..? I am checking the method with least time frame of execution.

Thanks in advance,
Leo Prince

On Mon, Nov 27, 2017 at 2:58 PM, Daniel Collins <danwcollins@gmail.com>
wrote:

> Leo, the general rule of thumb here is that the Solr index should *not* be
> your main document store.  It is the index to your document store, but if
> it needs to be re-indexed, you should use your document store as the place
> to index from.
>
> Your index will not have the full source data (unless ALL your fields have
> stored=true, which suggests you are using it as a document store, see my
> first point), so you should look to where you index your data from, that is
> what should drive your new index.
>
> On 27 November 2017 at 07:16, Leo Prince <leoprince.discussions@gmail.com>
> wrote:
>
> > Hi Shawn,
> >
> > Thanks for the help.
> >
> > I hate to burst your bubble here ... but 4 million docs is pretty small
> for
> > > a Solr index.  I have one index that's a hundred times larger, and
> there
> > > are people with *billions* of documents in SolrCloud.
> > >
> >
> > Sorry I missed a "0" there. It's actually 40 Millions, still according to
> > you, it's still a small sized index. 😊😊
> >
> >
> > You would need to keep the schema the same for the upgrade, except that
> you
> > > would need to disable docValues on some of your fields to get rid of
> the
> > > error you encountered.  You won't be able to take advantage of some of
> > the
> > > new capability in the new version unless you re-engineer your
> > config/schema
> > > and reindex.
> > >
> >
> > Thanks.. Got your point.
> >
> >
> >
> > >
> > > Upgrading an index, especially through three major versions, is
> generally
> > > not recommended.  I always reindex when upgrading Solr, especially to a
> > new
> > > major version, because Solr evolves quickly.
> > >
> >
> > What method do you actually follow to re-index upon Solr major upgrade..?
> >
> > I think I should do re-index since I am upgrading 3 major versions at
> once.
> > What is best method to re-index..? At present, I am planning to re-index
> by
> > SELECT from previous version (4.10.2) and then UPDATE into latest
> > version(7.1.0). Any other better thoughts to re-index..?
> >
> > Thanks in advance,
> > Leo Prince.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message