lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-6969) Exception in reading SortedDocValues
Date Sun, 10 Jan 2016 05:44:39 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-6969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

John Wang updated LUCENE-6969:
------------------------------
    Description: 
While reading SortedDocValues, I am getting the following stacktrace:
{noformat}
Exception in thread "main" java.lang.IndexOutOfBoundsException
	at java.nio.Buffer.checkIndex(Buffer.java:546)
	at java.nio.DirectByteBuffer.getShort(DirectByteBuffer.java:590)
	at org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.readShort(ByteBufferIndexInput.java:443)
	at org.apache.lucene.util.packed.DirectReader$DirectPackedReader16.get(DirectReader.java:185)
	at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$2.get(Lucene54DocValuesProducer.java:502)
	at org.apache.lucene.util.LongValues.get(LongValues.java:45)
	at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$7.getOrd(Lucene54DocValuesProducer.java:800)
{noformat}

I am not able to reproduce with a unit test. However, I am able to consistently reproduce
it with my data. I think this is some sort of off-by-one error caused with my index.

I am happy to provide my index for reproduce this offline.

Here is the code snippet:
{noformat}
public static void main(String[] args) throws Exception {
    File idx = new File("/tmp/myidx");
    int id = 719265;
    Path idxPath = FileSystems.getDefault().getPath(idx.getAbsolutePath());
    FSDirectory dir = FSDirectory.open(idxPath);
    DirectoryReader dreader = DirectoryReader.open(dir);
    LeafReaderContext ctx = dreader.leaves().get(0);
        
    SortedDocValues docVal = ctx.reader().getSortedDocValues("host");
    int ord = docVal.getOrd(id);    
    System.out.println("ord: " + ord);
    
  }
{noformat}

My index has 1 segment and 5M docs.

  was:
While reading SortedDocValues, I am getting the following stacktrace:
{noformat}
Exception in thread "main" java.lang.IndexOutOfBoundsException
	at java.nio.Buffer.checkIndex(Buffer.java:546)
	at java.nio.DirectByteBuffer.getShort(DirectByteBuffer.java:590)
	at org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.readShort(ByteBufferIndexInput.java:443)
	at org.apache.lucene.util.packed.DirectReader$DirectPackedReader16.get(DirectReader.java:185)
	at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$2.get(Lucene54DocValuesProducer.java:502)
	at org.apache.lucene.util.LongValues.get(LongValues.java:45)
	at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$7.getOrd(Lucene54DocValuesProducer.java:800)
{noformat}

I am not able to reproduce with a unit test. However, I am able to consistently reproduce
it with my data. I think this is some sort of off-by-one error caused with my index.

I am happy to provide my index for reproduce this offline.

Here is the code snippet:
{noformat}
public static void main(String[] args) throws Exception {
    File idx = new File("/tmp/rapid/data/1448008627000_1448120881000");
    int id = 719265;
    Path idxPath = FileSystems.getDefault().getPath(idx.getAbsolutePath());
    FSDirectory dir = FSDirectory.open(idxPath);
    DirectoryReader dreader = DirectoryReader.open(dir);
    LeafReaderContext ctx = dreader.leaves().get(0);
        
    SortedDocValues docVal = ctx.reader().getSortedDocValues("host");
    int ord = docVal.getOrd(id);    
    System.out.println("ord: " + ord);
    
  }
{noformat}

My index has 1 segment and 5M docs.


> Exception in reading SortedDocValues
> ------------------------------------
>
>                 Key: LUCENE-6969
>                 URL: https://issues.apache.org/jira/browse/LUCENE-6969
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 5.4
>            Reporter: John Wang
>
> While reading SortedDocValues, I am getting the following stacktrace:
> {noformat}
> Exception in thread "main" java.lang.IndexOutOfBoundsException
> 	at java.nio.Buffer.checkIndex(Buffer.java:546)
> 	at java.nio.DirectByteBuffer.getShort(DirectByteBuffer.java:590)
> 	at org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.readShort(ByteBufferIndexInput.java:443)
> 	at org.apache.lucene.util.packed.DirectReader$DirectPackedReader16.get(DirectReader.java:185)
> 	at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$2.get(Lucene54DocValuesProducer.java:502)
> 	at org.apache.lucene.util.LongValues.get(LongValues.java:45)
> 	at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$7.getOrd(Lucene54DocValuesProducer.java:800)
> {noformat}
> I am not able to reproduce with a unit test. However, I am able to consistently reproduce
it with my data. I think this is some sort of off-by-one error caused with my index.
> I am happy to provide my index for reproduce this offline.
> Here is the code snippet:
> {noformat}
> public static void main(String[] args) throws Exception {
>     File idx = new File("/tmp/myidx");
>     int id = 719265;
>     Path idxPath = FileSystems.getDefault().getPath(idx.getAbsolutePath());
>     FSDirectory dir = FSDirectory.open(idxPath);
>     DirectoryReader dreader = DirectoryReader.open(dir);
>     LeafReaderContext ctx = dreader.leaves().get(0);
>         
>     SortedDocValues docVal = ctx.reader().getSortedDocValues("host");
>     int ord = docVal.getOrd(id);    
>     System.out.println("ord: " + ord);
>     
>   }
> {noformat}
> My index has 1 segment and 5M docs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message