lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Kan <solrexp...@gmail.com>
Subject Re: SegmentInfos exposed to /admin/luke
Date Mon, 08 Dec 2014 11:23:18 GMT
Hi Alexey,

In GUI luke there is an option to "Just expunge deleted docs without
re-merging". In case you want to give it a try.

Dmitry

On Wed, Dec 3, 2014 at 1:35 PM, Alexey Kozhemiakin <
Alexey_Kozhemiakin@epam.com> wrote:

> Dear All,
>
> We have a high percentage of deleted docs which do not go away because
> there are several huge ancient segments that do not merge with anything
> else naturally. Our use case in constant reindexing of same data - ~100 gb,
> 12 000 000 real records, 20 000 000 total records in index, which is ~80%
> deletes.
>
> We plan to deal with situation by playing with mergeFactor,
> reclaimDeletesWeight and maxSegmentSizeMB settings to optimize for our
> re-indexing rate and data size.
> And in order to do it with eyes-opened we want to see a picture similar to
> http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
> with columns of segment size and %of deletes.
> The plan is to expose SegmentInfos via /admin/luke handler and draw column
> bars in Solr admin.
>
> Is there an easier way to achieve that? Even in raw Luke we didn't' found
> these data.
>
> We'd be happy to push the changes to Solr afterwards.
>
>
> Thank you,
> Alexey Kozhemiakin
>
>


-- 
Dmitry Kan
Luke Toolbox: http://github.com/DmitryKey/luke
Blog: http://dmitrykan.blogspot.com
Twitter: http://twitter.com/dmitrykan
SemanticAnalyzer: www.semanticanalyzer.info

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message