lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Green <eelstretch...@gmail.com>
Subject Re: Lucene indexing throughput (and Mike's lucenebench charts)
Date Thu, 14 Apr 2016 20:25:20 GMT
As someone who runs Lucene on big hardware, I'd be very interested to see
the tuning parameters when you do get a chance..

On Thu, Apr 14, 2016 at 3:41 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> Yes, dual 2699 v3, with 256 GB of RAM, yet indexing throughput somehow
> got slower :)  I haven't re-tuned indexing threads, IW buffer size yet
> for this new hardware ...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Thu, Apr 14, 2016 at 2:09 PM, Ishan Chattopadhyaya
> <ichattopadhyaya@gmail.com> wrote:
> > Wow, 72 cores? That sounds astounding. Are they dual Xeon E5 2699 v3 CPUs
> > with 18 cores each, with hyperthreading = 18*2*2=72 threads?
> >
> > On Thu, Apr 14, 2016 at 11:33 PM, Dawid Weiss <dawid.weiss@gmail.com>
> wrote:
> >
> >> The GC change is after this:
> >>
> >> BJ (2015-12-02): Upgrade to beast2 (72 cores, 256 GB RAM)
> >>
> >> which leads me to believe these results are not comparable (different
> >> machines, architectures, disks, CPUs perhaps?).
> >>
> >> Dawid
> >>
> >>
> >> On Thu, Apr 14, 2016 at 7:13 PM, Otis Gospodnetić
> >> <otis.gospodnetic@gmail.com> wrote:
> >> > Hi,
> >> >
> >> > I was looking at Mike's
> >> > http://home.apache.org/~mikemccand/lucenebench/indexing.html secretly
> >> > hoping to spot some recent improvements in indexing throughput.... but
> >> > instead it looks like:
> >> >
> >> > * indexing throughput hasn't really gone up in the last ~5 years
> >> > * indexing was faster in 2014, but then dropped to pre-2014 speed in
> >> early
> >> > 2015
> >> > * indexing rate dropped some more in early 2016, and that seems to
> >> roughly
> >> > correlate to a *big* jump in Young GC in late 2015
> >> >
> >> > Does anyone know what happened in late 2015 that causes that big
> Young GC
> >> > jump?
> >> > Or does that big jump just look scary in that chart, but is not
> actually
> >> a
> >> > big concern in practice?
> >> >
> >> > Thanks,
> >> > Otis
> >> > --
> >> > Monitoring - Log Management - Alerting - Anomaly Detection
> >> > Solr & Elasticsearch Consulting Support Training -
> http://sematext.com/
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>


-- 
Stephen Green
http://thesearchguy.wordpress.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message