lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: JDK 8 great performance increase on ByteBuffer read performance!?
Date Mon, 07 Apr 2014 21:47:42 GMT
It was a typo :)  A fun typo, I'll agree!

Mike McCandless

http://blog.mikemccandless.com


On Mon, Apr 7, 2014 at 5:44 PM, Erick Erickson <erickerickson@gmail.com> wrote:
> Right, but what's this "jaba7"? Some inside Star Wars joke :)
>
>
> On Mon, Apr 7, 2014 at 1:49 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
>> Hi Mike,
>>
>> hehe, as expected. I think, we need to look at stuff like off-heap filters: https://issues.apache.org/jira/browse/LUCENE-5052
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>>> -----Original Message-----
>>> From: Michael McCandless [mailto:lucene@mikemccandless.com]
>>> Sent: Monday, April 07, 2014 10:19 PM
>>> To: Lucene/Solr dev
>>> Subject: Re: JDK 8 great performance increase on ByteBuffer read
>>> performance!?
>>>
>>> Here are the results: https://paste.apache.org/cKRW
>>>
>>> Baseline is 1.7 u60 ea, comparison is 1.8.0, index is full 33.3 "Wiki medium"
>>> (English) docs.
>>>
>>> Net/net Java 8 seems to love fuzziness :)
>>>
>>> Mike McCandless
>>>
>>> http://blog.mikemccandless.com
>>>
>>>
>>> On Mon, Apr 7, 2014 at 1:01 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
>>> > Thanks Mike.
>>> >
>>> > In fact this improvement would help stuff like Yonik's Off-Heap Filters
and
>>> Bitsets, because that code is using Unsafe directly to work around
>>> ByteBuffers overhead. If those getByte()/getLong() calls are really faster, we
>>> can think about providing those off-heap data structures based on
>>> DirectByteBuffer in the future. I am thinking about CachedFilter, if we have
a
>>> FixedOffHeapBitSet, or using DirectBuffers for FieldCache or DocValues. The
>>> new FieldCache interface based on DocValues without direct array acess
>>> makes this easy possible.
>>> >
>>> > Uwe
>>> >
>>> > -----
>>> > Uwe Schindler
>>> > H.-H.-Meier-Allee 63, D-28213 Bremen
>>> > http://www.thetaphi.de
>>> > eMail: uwe@thetaphi.de
>>> >
>>> >
>>> >> -----Original Message-----
>>> >> From: Michael McCandless [mailto:lucene@mikemccandless.com]
>>> >> Sent: Monday, April 07, 2014 6:55 PM
>>> >> To: Lucene/Solr dev
>>> >> Subject: Re: JDK 8 great performance increase on ByteBuffer read
>>> >> performance!?
>>> >>
>>> >> In fact I just recently tested Java 8 GA vs Java 7 (I think 1.7 u25)
>>> >> and I think there were small improvements, curiously especially to
>>> >> span and sloppy phrase queries.
>>> >>
>>> >> Since we mostly read big byte[] blocks I think gains to
>>> >> ByteBuffer.getXXX won't help us that much.  But I'll retest, with 1.7
>>> >> u60 and report back.
>>> >>
>>> >> Mike McCandless
>>> >>
>>> >> http://blog.mikemccandless.com
>>> >>
>>> >>
>>> >> On Mon, Apr 7, 2014 at 11:59 AM, Uwe Schindler <uwe@thetaphi.de>
>>> wrote:
>>> >> > Hi,
>>> >> >
>>> >> > On the Hotspot mailing list was the following post by Cleber
>>> >> > Muramoto. It
>>> >> might be interesting to check MMapDirectory in Java 8!
>>> >> >  Mike, do you maybe setup another luceneutil test on Lucene on
>>> >> > Beast
>>> >> with JDK 8 GA? Can we do a comparison between JDK 7u60 and JDK 8 GA?
>>> >> It would be very interesting, because one reason some people want to
>>> >> have the native mmap variants because of the additional slowdown
>>> >> caused by the ByteBuffer wrapping.
>>> >> >
>>> >> > Uwe
>>> >> >
>>> >> > === snip ===
>>> >> > Hello, I'm curious to know if there has been any low-level
>>> >> > optmizations
>>> >> regarding direct buffers getXXX methods on JDK8 and, if they're
>>> >> planned to be integrated in JDK7, if applicable. I googled and took
a
>>> >> look at the bug database but I couldn't find anything related.
>>> >> >
>>> >> > I have a microbenchmark that does millions of iterations
>>> >> > serializing and
>>> >> deserializing objects to/from ByteBuffers and I noticed that read
>>> >> performance on JDK 8 has increased by almost 45% in comparison to
>>> >> earliear JDK 7 releases in the serial case and more than 200% on the
>>> concurrent case!
>>> >> I think that the concurrent test is perhaps benefiniting from newer
>>> >> ForkJoinPool/concurrency code, but the difference in the serial case
>>> >> is still very large!
>>> >> >
>>> >> > Bellow are the VM arguments that I'm using for the test:
>>> >> >
>>> >> > $JAVA -server -XX:+UseParallelGC \
>>> >> > -XX:+UseLargePages -XX:MaxDirectMemorySize=10G -Xmx1g
>>> >> > -XX:MaxInlineSize=256 \ -XX:+UnlockDiagnosticVMOptions
>>> >> > -XX:+PrintInlining -XX:+LogCompilation
>>> >> >
>>> >> > JDK8 seems to generate much more compiling information than the
>>> >> previous versions, but I wasn't able to find any indicators for such
>>> >> huge performance difference.
>>> >> >
>>> >> > Here are the test results (collected with the diagnostic flags
off,
>>> >> > on a HP
>>> >> > G7 48 CPU box).
>>> >> >
>>> >> > Concurrent Reads:
>>> >> >
>>> >> > 503K   Reads/s|Writes/s 161K (jdk7U10)
>>> >> > 940K   Reads/s|Writes/s 165K (jdk7U40)
>>> >> > 956K   Reads/s|Writes/s 159K (jdk7U60 EA)
>>> >> > 1644K Reads/s|Writes/s 172K (jdk8-more than 3x faster than U10!)
>>> >> >
>>> >> > Serial Reads:
>>> >> >
>>> >> > 137K Reads/s|Writes/s 146K
>>> >> > 145K Reads/s|Writes/s 145K
>>> >> > 143K Reads/s|Writes/s 155K
>>> >> > 198K Reads/s|Writes/s 172K
>>> >> >
>>> >> > Upon deserialization the test does a lot of short-lived
>>> >> > allocations, but GC
>>> >> reports show very similar results for every JDK versions used in the
>>> >> test (about ~500 Young Gen Collections and ~1800ms spent by PS
>>> Scavenge).
>>> >> >
>>> >> > Anyway, great work!!! I hope I can migrate to JDK8 as soon as possible.
>>> >> >
>>> >> > Regards
>>> >> >
>>> >> > Cleber
>>> >> >
>>> >> > -----
>>> >> > Uwe Schindler
>>> >> > H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
>>> >> > eMail: uwe@thetaphi.de
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > ---------------------------------------------------------------------
>>> >> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>> >> > additional commands, e-mail: dev-help@lucene.apache.org
>>> >> >
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>> additional
>>> >> commands, e-mail: dev-help@lucene.apache.org
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message