lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Bower <sbo...@alcyon.net>
Subject Re: Performance question on Spatial Search
Date Tue, 30 Jul 2013 21:10:41 GMT
Very good read... Already using MMap... verified using pmap and vsz from
top..

not sure what you mean by good hit raitio?

Here are the stacks...

  Name Time (ms) Own Time (ms)
org.apache.lucene.search.MultiTermQueryWrapperFilter.getDocIdSet(AtomicReaderContext,
Bits) 300879 203478
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.nextDoc()
45539 19
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader$BlockDocsEnum.refillDocs()
45519 40
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.readVIntBlock(IndexInput,
int[], int[], int, boolean) 24352 0
org.apache.lucene.store.DataInput.readVInt() 24352 24352
org.apache.lucene.codecs.lucene41.ForUtil.readBlock(IndexInput, byte[],
int[]) 21126 14976
org.apache.lucene.store.ByteBufferIndexInput.readBytes(byte[], int, int)
6150 0              java.nio.DirectByteBuffer.get(byte[], int, int)
6150 0
java.nio.Bits.copyToArray(long, Object, long, long, long) 6150 6150
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.docs(Bits,
DocsEnum, int) 35342 421
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.decodeMetaData()
34920 27939
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.nextTerm(FieldInfo,
BlockTermState) 6980 6980
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.next()
14129 1053
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadNextFloorBlock()
5948 261
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock()
5686 199
org.apache.lucene.store.ByteBufferIndexInput.readBytes(byte[], int, int)
3606 0              java.nio.DirectByteBuffer.get(byte[], int, int)
3606 0
java.nio.Bits.copyToArray(long, Object, long, long, long) 3606 3606
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.readTermsBlock(IndexInput,
FieldInfo, BlockTermState) 1879 80
org.apache.lucene.store.ByteBufferIndexInput.readBytes(byte[], int, int)
1798 0                java.nio.DirectByteBuffer.get(byte[], int, int)
1798 0
java.nio.Bits.copyToArray(long, Object, long, long, long) 1798 1798
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.next()
4010 3324
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.nextNonLeaf()
685 685
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock()
3117 144
org.apache.lucene.store.ByteBufferIndexInput.readBytes(byte[], int, int)
1861 0            java.nio.DirectByteBuffer.get(byte[], int, int) 1861
0
java.nio.Bits.copyToArray(long, Object, long, long, long) 1861 1861
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.readTermsBlock(IndexInput,
FieldInfo, BlockTermState) 1090 19
org.apache.lucene.store.ByteBufferIndexInput.readBytes(byte[], int, int)
1070 0              java.nio.DirectByteBuffer.get(byte[], int, int)
1070 0
java.nio.Bits.copyToArray(long, Object, long, long, long) 1070 1070
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.initIndexInput()
20 0            org.apache.lucene.store.ByteBufferIndexInput.clone()
20 0
org.apache.lucene.store.ByteBufferIndexInput.clone() 20 0
org.apache.lucene.store.ByteBufferIndexInput.buildSlice(long, long) 20
0
org.apache.lucene.util.WeakIdentityMap.put(Object, Object) 20 0
org.apache.lucene.util.WeakIdentityMap$IdentityWeakReference.<init>(Object,
ReferenceQueue) 20 0
java.lang.System.identityHashCode(Object) 20 20
org.apache.lucene.index.FilteredTermsEnum.docs(Bits, DocsEnum, int)
1485 527
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.docs(Bits,
DocsEnum, int) 957 0
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.decodeMetaData()
957 513
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.nextTerm(FieldInfo,
BlockTermState) 443 443
org.apache.lucene.index.FilteredTermsEnum.next() 874 324
org.apache.lucene.search.NumericRangeQuery$NumericRangeTermsEnum.accept(BytesRef)
368 0
org.apache.lucene.util.BytesRef$UTF8SortedAsUnicodeComparator.compare(Object,
Object) 368 368
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.next()
160 0
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadNextFloorBlock()
160 0
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock()
160 0
org.apache.lucene.store.ByteBufferIndexInput.readBytes(byte[], int, int) 120
0
org.apache.lucene.codecs.lucene41.Lucene41PostingsReader.readTermsBlock(IndexInput,
FieldInfo, BlockTermState) 39 0
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.seekCeil(BytesRef,
boolean) 19 0
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum$Frame.loadBlock()
19 0
org.apache.lucene.codecs.BlockTreeTermsReader$FieldReader$SegmentTermsEnum.initIndexInput()
19 0              org.apache.lucene.store.ByteBufferIndexInput.clone()
19 0
org.apache.lucene.store.ByteBufferIndexInput.clone() 19 0
org.apache.lucene.store.ByteBufferIndexInput.buildSlice(long, long) 19
0
org.apache.lucene.util.WeakIdentityMap.put(Object, Object) 19 0
org.apache.lucene.util.WeakIdentityMap$IdentityWeakReference.<init>(Object,
ReferenceQueue) 19 0
java.lang.System.identityHashCode(Object) 19 19
org.apache.lucene.util.FixedBitSet.<init>(int) 28 28


On Tue, Jul 30, 2013 at 4:18 PM, Mikhail Khludnev <
mkhludnev@griddynamics.com> wrote:

> On Tue, Jul 30, 2013 at 12:45 AM, Steven Bower <smb-apache@alcyon.net
> >wrote:
>
> >
> > - Most of my time (98%) is being spent in
> > java.nio.Bits.copyToByteArray(long,Object,long,long) which is being
>
>
> Steven, please
> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html.my
> benchmarking experience shows that NIO is a turtle, absolutely.
>
> also, are you sure that fq=(vid:86XXX73 OR vid:86XXX20 ..... has good hit
> ratio? otherwise it's a  well known beast.
>
> could you also show deeper stack, to make sure what causes to excessive
> reading?
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
>  <mkhludnev@griddynamics.com>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message