lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From will <wmartin...@gmail.com>
Subject Re: index size growing while deleting
Date Thu, 05 Nov 2015 13:52:15 GMT
Hi Rob:

Do you understand how deletes work and how an index is compacted?

There's some configuration/runtime activities you don't mention.... And 
you make testing process sound like a mirror of production? (Including 
configuration?)


-will

On 11/5/15 7:33 AM, Rob Audenaerde wrote:
> Hi all,
>
> I'm currently investigating an issue we have with our index. It keeps
> getting bigger, and I don't het why.
>
> Here is our use case:
>
> We index a database of about 4 million records; spread over a few hundred
> tables. The data consists of a mix of text, dates, numbers etc. We also add
> all these fields as facets.
> Each night we delete about 90% of the data, which in testing reduces the
> index size significantly.
> We store the data as StoredFields as well, to prevent having to access the
> database at all.
> We use FloatAssociatedFacet fields for the facets.
>
>
> In production however, it seems the index is only growing, up to 71 GB for
> these records for a month of running.
>
> It seems that lucene's index in just getting bigger there.
>
> We use lucene 5.3 on CentOS, java 8 64 bit.
>
> The taxonomy-index does not grow significantly.
>
> How should I go about checking what is wrong?
>
> Thanks!
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message