lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Rafalovitch <arafa...@gmail.com>
Subject Re: Frequent deletions
Date Thu, 01 Jan 2015 16:19:43 GMT
Is there a specific list of which data structures are "sparce" and
"non-sparce" for Lucene and Solr (referencing G+ post)? I imagine this
is obvious to low-level hackers, but could actually be nice to
summarize it somewhere for troubleshooting.

Regards,
   Alex.
----
Sign up for my Solr resources newsletter at http://www.solr-start.com/


On 1 January 2015 at 05:22, Michael McCandless
<lucene@mikemccandless.com> wrote:
> Also see this G+ post I wrote up recently showing how %tg deletions
> changes over time for an "every add also deletes a previous document"
> stress test: https://plus.google.com/112759599082866346694/posts/MJVueTznYnD
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Wed, Dec 31, 2014 at 12:21 PM, Erick Erickson
> <erickerickson@gmail.com> wrote:
>> It's usually not necessary to optimize, as more indexing happens you
>> should see background merges happen that'll reclaim the space, so I
>> wouldn't worry about it unless you're seeing actual problems that have
>> to be addressed. Here's a great visualization of the process:
>>
>> http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html
>>
>> See especially the third video, "TieredMergePolicy" which is the default.
>>
>> If you insist, however, try a commit with expungeDeletes=true
>>
>> and if that isn't enough, try an optimize call
>> you can issue a "force merge" (aka optimize)  command from the URL (Or
>> cUrl etc) as:
>> http://localhost:8983/solr/techproducts/update?optimize=true
>>
>> But please don't do this unless it's absolutely necessary. You state
>> that you have "frequent deletions", but eventually this shoul dall
>> happen in the background. Optimize is a fairly expensive operation and
>> should be used judiciously.
>>
>> Best,
>> Erick
>>
>> On Wed, Dec 31, 2014 at 1:32 AM, ig01 <inna.geller@elbitsystems.com> wrote:
>>> Hello,
>>> We perform frequent deletions from our index, which greatly increases the
>>> index size.
>>> How can we perform an optimization in order to reduce the size.
>>> Please advise,
>>> Thanks.
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://lucene.472066.n3.nabble.com/Frequent-deletions-tp4176689.html
>>> Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message