lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rishi Easwaran <>
Subject Re: Solr Cloud reclaiming disk space from deleted documents
Date Mon, 04 May 2015 10:55:21 GMT
Sadly with the size of our complex, spiting and adding more HW is not a viable long term solution.

 I guess the options we have are to run optimize regularly and/or become aggressive in our
merges proactively even before solr cloud gets into this situation.



-----Original Message-----
From: Gili Nachum <>
To: solr-user <>
Sent: Mon, Apr 27, 2015 4:18 pm
Subject: Re: Solr Cloud reclaiming disk space from deleted documents

To prevent it from re occurring you could monitor index size and once above
certain size threshold add another machine and split the shard between
and new machine.
On Apr 20, 2015 9:10 PM, "Rishi Easwaran"
<> wrote:

> So is there anything that can be done from
a tuning perspective, to
> recover a shard that is 75%-90% full, other that get
rid of the index and
> rebuild the data?
>  Also to prevent this issue from
re-occurring, looks like we need make our
> system aggressive with segment
merges using lower merge factor
> Thanks,
> Rishi.
-----Original Message-----
> From: Shawn Heisey <>
> To:
solr-user <>
> Sent: Mon, Apr 20, 2015 11:25 am
Subject: Re: Solr Cloud reclaiming disk space from deleted documents
> On
4/20/2015 8:44 AM, Rishi Easwaran wrote:
> > Yeah I noticed that. Looks like
optimize won't work since on some disks we are already pretty full.
> > Any
thoughts on increasing/decreasing <mergeFactor>10</mergeFactor>  or
ConcurrentMergeScheduler to make solr do merges faster.
> You don't have to
> an optimize to need 2x disk space.  Even normal
> merging, if it happens
> right, can require the same disk space as a
> full optimize.  Normal
> operation requires that you have enough
> space for your index to reach
at least
> double size on occasion.
> Higher merge factors are better for
indexing speed,
> because merging
> happens less frequently.  Lower merge
factors are better for
> query
> speed, at least after the merging finishes,
because merging happens
> more
> frequently and there are fewer total segments
at any given moment.
> During a merge, there is so much I/O that query speed
is often
> negatively
> affected.
> Thanks,
> Shawn


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message