lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zheng Lin Edwin Yeo <edwinye...@gmail.com>
Subject Re: Memory Usage increases by a lot during and after optimization .
Date Fri, 01 Jan 2016 03:03:39 GMT
Hi Yonik,

Yes, the plan is to do the optimizing at night after indexing, when there
are lesser user who will use the system.

But the problem I'm facing now is that during optimizing, the memory usage
of the server hit the maximum of 64GB, and I believe the optimization could
not be completed fully as there is not enough memory, so when I check the
index again, it says that it is not optimized. Before the optimization, the
memory usage was less than 16GB, so the optimization actually uses up more
than 48GB of memory.

Is it normal for an index size of 200GB to use up so much memory during
optimization?

Regards,
Edwin


On 30 December 2015 at 11:49, Yonik Seeley <yseeley@gmail.com> wrote:

> Some people also want to control when major segment merges happen, and
> optimizing at a known time helps prevent a major merge at an unknown
> time (which can be equivalent to an optimize/forceMerge).
>
> The benefits of optimizing (and having fewer segments to search
> across) will vary depending on the requests.
> Normal full-text searches will see little benefit (merging a few terms
> across many segments is not expensive), while other operations that
> need to deal with many terms, like faceting, may see bigger speedups.
>
> -Yonik
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message