lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Krishnamurthy, Kannan" <Kannan.Krishnamur...@cengage.com>
Subject Re: Lucene 6.1.0 index upgrade
Date Thu, 09 Nov 2017 23:46:18 GMT
Hi Mike, 

It doesn’t work either, it throws “Cannot merge a segment that has been created with major
version 6 into this index which has been created by major version 7" message. Let me try it
after an upgrade as well. Same code below, 

	@Test
	public void mergeUsingCodecReader() throws IOException {
		  DirectoryReader ir = DirectoryReader.open(lucene610Dir);
	  	  SegmentReader air = (SegmentReader)ir.leaves().get(0).reader();
	    
		IndexWriterConfig config = newIndexWriterConfig();
		IndexWriter w = new IndexWriter(newEmptyDirectory("target"), config);
		exception.expect(IllegalArgumentException.class);
		exception.expectMessage("Cannot merge a segment that has been created with major version
6 into this index which has been created by major version 7");
		
		w.addIndexes(air);
		w.forceMerge(1);
		w.close();
	}
Thanks, 
Kannan

On 11/9/17, 5:07 PM, "Michael McCandless" <lucene@mikemccandless.com> wrote:

    Maybe try the addIndexes method that takes CodecReader[] instead?  It will
    be somewhat slower, since it fully reads and writes the index, but it
    should work across major versions?
    
    Mike McCandless
    
    http://blog.mikemccandless.com
    
    On Thu, Nov 9, 2017 at 3:59 PM, Krishnamurthy, Kannan <
    Kannan.Krishnamurthy@cengage.com> wrote:
    
    > Greetings.
    >
    >
    > We are currently using Lucene 6.1.0, our typical index size is ~ 200GB and
    > we are planning to move to Lucene 7.0.0.
    >
    >
    > Our daily indexing workflow currently uses IndexWriter.addIndexes() to add
    > an index(this will be on Lucene 7.0.0) to a copy of the of the 200GB index.
    > Further it calls IndexWriter.forceMerge(1) to create a single segment. The
    > addIndexes() now doesn't support different versions of index.
    >
    > We are facing an issue in upgrading to Lucene 7.0.0 due to this
    > limitation. We couldn't afford to do full reindex as there about 50
    > different indexes ~ 200GB each.
    >
    >
    > We tried using the LuceneUpgrade tool to upgrade our index to latest
    > version, however the IndexCreatedVersionMajor is still LUCENE_6_1_0.major,
    > This is preventing us calling IndexWriter.addIndexes() post upgrade.
    >
    >
    > I am wondering what are our options to upgrade without doing a full
    > reindex and to continue to use IndexWriter.addIndexes() post upgrade.
    >
    > The code snippet below shows the different api calls we make.
    >
    >
    >
    > @Test
    > public void addIndexWith7SegmentToDirectoryOnUpgradedSegment() throws
    > IOException {
    > IndexUpgrader upgrader = new IndexUpgrader(lucene610Dir); // lucene 6.1.0
    > directory 200 GB
    > upgrader.upgrade();
    >
    > IndexWriter w = new IndexWriter(lucene610Dir, newIndexWriterConfig());
    > exception.expect(IllegalArgumentException.class);
    > exception.expectMessage(
    > CoreMatchers.containsString("Cannot use addIndexes(Directory) with
    > indexes that have been created by a different Lucene version"));
    > w.addIndexes(lucene700Dir); // lucene 7.0.0 daily index
    > w.forceMerge(1);
    > w.close();
    > }
    >
    > Let me know if you need more details.
    >
    >
    > Many thanks
    >
    > Kannan.
    >
    >
    

Mime
View raw message