lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "will martin" <wmartin...@gmail.com>
Subject RE: Lucene 5 : any merge performance metrics compared to 4.x?
Date Tue, 29 Sep 2015 16:08:32 GMT
So, if its new, it adds to pre-existing time? So it is a cost that needs to be understood I
think.

 

And, I'm really curious, what happens to the result of the post merge checkIntegrity IFF (if
and only if) there was corruption pre-merge: I mean if you let it merge anyway could you get
a false positive for integrity?  [see the concept of lazy-evaluation]

 

These are, imo, the kinds of engineering questions Selva's post raised in my triage mode of
the scenario.

 

 

-----Original Message-----
From: Adrien Grand [mailto:jpountz@gmail.com] 
Sent: Tuesday, September 29, 2015 8:46 AM
To: java-user@lucene.apache.org
Subject: Re: Lucene 5 : any merge performance metrics compared to 4.x?

 

Indeed this is new but I'm a bit surprised this is the source of your issues as it should
be much faster than the merge itself. I don't understand your proposal to check the index
after merge: the goal is to make sure that we do not propagate corruptions so it's better
to check the index before the merge starts so that we don't even try to merge if there are
corruptions?

 

Le mar. 15 sept. 2015 à 00:40, Selva Kumar < <mailto:selva.kumar.at.work@gmail.com>
selva.kumar.at.work@gmail.com> a écrit :

 

> it appears Lucene 5.2 index merge is running checkIntegrity on 

> existing index prior to merging additional indices.

> This seems to be new.

> 

> We have an existing checkIndex but this is run post index merge.

> 

> Two follow up questions :

> * Is there way to turn off built-in checkIntegrity? Just for my understand.

> No plan to turn this off.

> * Is running checkIntegrity prior to index merge better than running 

> post merge?

> 

> 

> On Mon, Sep 14, 2015 at 12:24 PM, Selva Kumar < 

>  <mailto:selva.kumar.at.work@gmail.com> selva.kumar.at.work@gmail.com

> > wrote:

> 

> > We observe some merge slowness after we migrated from 4.10 to 5.2.

> > Is this expected? Any new tunable merge parameters in Lucene 5 ?

> >

> > -Selva

> >

> >

> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message