lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antonio De Miguel <deveto...@gmail.com>
Subject Re: High disk write usage
Date Wed, 05 Jul 2017 15:08:41 GMT
thanks Markus!

We already have SSD.

About changing topology.... we probed yesterday with 10 shards, but system
goes more inconsistent than with the current topology (5x10). I dont know
why... too many traffic perhaps?

About merge factor.. we set default configuration for some days... but when
a merge occurs system overload. We probed with mergefactor of 4 to improbe
query times and trying to have smaller merges.

2017-07-05 16:51 GMT+02:00 Markus Jelsma <markus.jelsma@openindex.io>:

> Try mergeFactor of 10 (default) which should be fine in most cases. If you
> got an extreme case, either create more shards and consider better hardware
> (SSD's)
>
> -----Original message-----
> > From:Antonio De Miguel <devetonez@gmail.com>
> > Sent: Wednesday 5th July 2017 16:48
> > To: solr-user@lucene.apache.org
> > Subject: Re: High disk write usage
> >
> > Thnaks a lot alessandro!
> >
> > Yes, we have very big physical dedicated machines, with a topology of 5
> > shards and10 replicas each shard.
> >
> >
> > 1. transaction log files are increasing but not with this rate
> >
> > 2.  we 've probed with values between 300 and 2000 MB... without any
> > visible results
> >
> > 3.  We don't use those features
> >
> > 4. No.
> >
> > 5. I've probed with low and high mergefacors and i think that is  the
> point.
> >
> > With low merge factor (over 4) we 've high write disk rate as i said
> > previously
> >
> > with merge factor of 20, writing disk rate is decreasing, but now, with
> > high qps rates (over 1000 qps) system is overloaded.
> >
> > i think that's the expected behaviour :(
> >
> >
> >
> >
> > 2017-07-05 15:49 GMT+02:00 alessandro.benedetti <a.benedetti@sease.io>:
> >
> > > Point 2 was the ram Buffer size :
> > >
> > > *ramBufferSizeMB* sets the amount of RAM that may be used by Lucene
> > >          indexing for buffering added documents and deletions before
> they
> > > are
> > >          flushed to the Directory.
> > >          maxBufferedDocs sets a limit on the number of documents
> buffered
> > >          before flushing.
> > >          If both ramBufferSizeMB and maxBufferedDocs is set, then
> > >          Lucene will flush based on whichever limit is hit first.
> > >
> > > <ramBufferSizeMB>100</ramBufferSizeMB>
> > > <maxBufferedDocs>1000</maxBufferedDocs>
> > >
> > >
> > >
> > >
> > > -----
> > > ---------------
> > > Alessandro Benedetti
> > > Search Consultant, R&D Software Engineer, Director
> > > Sease Ltd. - www.sease.io
> > > --
> > > View this message in context: http://lucene.472066.n3.
> > > nabble.com/High-disk-write-usage-tp4344356p4344386.html
> > > Sent from the Solr - User mailing list archive at Nabble.com.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message