lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: JDK 8 great performance increase on ByteBuffer read performance!?
Date Mon, 07 Apr 2014 20:18:35 GMT
Here are the results: https://paste.apache.org/cKRW

Baseline is 1.7 u60 ea, comparison is 1.8.0, index is full 33.3 "Wiki
medium" (English) docs.

Net/net Java 8 seems to love fuzziness :)

Mike McCandless

http://blog.mikemccandless.com


On Mon, Apr 7, 2014 at 1:01 PM, Uwe Schindler <uwe@thetaphi.de> wrote:
> Thanks Mike.
>
> In fact this improvement would help stuff like Yonik's Off-Heap Filters and Bitsets,
because that code is using Unsafe directly to work around ByteBuffers overhead. If those getByte()/getLong()
calls are really faster, we can think about providing those off-heap data structures based
on DirectByteBuffer in the future. I am thinking about CachedFilter, if we have a FixedOffHeapBitSet,
or using DirectBuffers for FieldCache or DocValues. The new FieldCache interface based on
DocValues without direct array acess makes this easy possible.
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Michael McCandless [mailto:lucene@mikemccandless.com]
>> Sent: Monday, April 07, 2014 6:55 PM
>> To: Lucene/Solr dev
>> Subject: Re: JDK 8 great performance increase on ByteBuffer read
>> performance!?
>>
>> In fact I just recently tested Java 8 GA vs Java 7 (I think 1.7 u25) and I think
>> there were small improvements, curiously especially to span and sloppy
>> phrase queries.
>>
>> Since we mostly read big byte[] blocks I think gains to ByteBuffer.getXXX
>> won't help us that much.  But I'll retest, with 1.7
>> u60 and report back.
>>
>> Mike McCandless
>>
>> http://blog.mikemccandless.com
>>
>>
>> On Mon, Apr 7, 2014 at 11:59 AM, Uwe Schindler <uwe@thetaphi.de> wrote:
>> > Hi,
>> >
>> > On the Hotspot mailing list was the following post by Cleber Muramoto. It
>> might be interesting to check MMapDirectory in Java 8!
>> >  Mike, do you maybe setup another luceneutil test on Lucene on Beast
>> with JDK 8 GA? Can we do a comparison between JDK 7u60 and JDK 8 GA? It
>> would be very interesting, because one reason some people want to have
>> the native mmap variants because of the additional slowdown caused by the
>> ByteBuffer wrapping.
>> >
>> > Uwe
>> >
>> > === snip ===
>> > Hello, I'm curious to know if there has been any low-level optmizations
>> regarding direct buffers getXXX methods on JDK8 and, if they're planned to
>> be integrated in JDK7, if applicable. I googled and took a look at the bug
>> database but I couldn't find anything related.
>> >
>> > I have a microbenchmark that does millions of iterations serializing and
>> deserializing objects to/from ByteBuffers and I noticed that read
>> performance on JDK 8 has increased by almost 45% in comparison to earliear
>> JDK 7 releases in the serial case and more than 200% on the concurrent case!
>> I think that the concurrent test is perhaps benefiniting from newer
>> ForkJoinPool/concurrency code, but the difference in the serial case is still
>> very large!
>> >
>> > Bellow are the VM arguments that I'm using for the test:
>> >
>> > $JAVA -server -XX:+UseParallelGC \
>> > -XX:+UseLargePages -XX:MaxDirectMemorySize=10G -Xmx1g
>> > -XX:MaxInlineSize=256 \ -XX:+UnlockDiagnosticVMOptions
>> > -XX:+PrintInlining -XX:+LogCompilation
>> >
>> > JDK8 seems to generate much more compiling information than the
>> previous versions, but I wasn't able to find any indicators for such huge
>> performance difference.
>> >
>> > Here are the test results (collected with the diagnostic flags off, on
>> > a HP
>> > G7 48 CPU box).
>> >
>> > Concurrent Reads:
>> >
>> > 503K   Reads/s|Writes/s 161K (jdk7U10)
>> > 940K   Reads/s|Writes/s 165K (jdk7U40)
>> > 956K   Reads/s|Writes/s 159K (jdk7U60 EA)
>> > 1644K Reads/s|Writes/s 172K (jdk8-more than 3x faster than U10!)
>> >
>> > Serial Reads:
>> >
>> > 137K Reads/s|Writes/s 146K
>> > 145K Reads/s|Writes/s 145K
>> > 143K Reads/s|Writes/s 155K
>> > 198K Reads/s|Writes/s 172K
>> >
>> > Upon deserialization the test does a lot of short-lived allocations, but GC
>> reports show very similar results for every JDK versions used in the test
>> (about ~500 Young Gen Collections and ~1800ms spent by PS Scavenge).
>> >
>> > Anyway, great work!!! I hope I can migrate to JDK8 as soon as possible.
>> >
>> > Regards
>> >
>> > Cleber
>> >
>> > -----
>> > Uwe Schindler
>> > H.-H.-Meier-Allee 63, D-28213 Bremen
>> > http://www.thetaphi.de
>> > eMail: uwe@thetaphi.de
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> > additional commands, e-mail: dev-help@lucene.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message