lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jürgen Wagner (DVT)" <juergen.wag...@devoteam.com>
Subject Re: Frequent deletions
Date Sun, 11 Jan 2015 12:06:21 GMT
Maybe you should consider creating different generations of indexes and
not keep everything in one index. If the likelihood of documents being
deleted is rather high in, e.g., the first week or so, you could have
one index for the high-probability of deletion documents (the fresh
ones) and a second one for the potentially longer-lived documents.
Without knowing the temporal distribution of deletion probabilities, it
is hard to say what would be the ideal index topology.

Apart from that, I have made the experience that in some cases where
Solr would produce the notorious out-of-memory exceptions, Elasticsearch
seems to be a bit more robust. You may want to give it a try as well.

Best regards,
--Jürgen

On 11.01.2015 07:46, ig01 wrote:
> Thank you all for your response,
> The thing is that we have 180G index while half of it are deleted documents.
> We  tried to run an optimization in order to shrink index size but it
> crashes on ‘out of memory’ when the process reaches 120G.   
> Is it possible to optimize parts of the index? 
> Please advise what can we do in this situation.
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Frequent-deletions-tp4176689p4178700.html
> Sent from the Solr - User mailing list archive at Nabble.com.


-- 

Mit freundlichen Grüßen/Kind regards/Cordialement vôtre/Atentamente/С
уважением
*i.A. Jürgen Wagner*
Head of Competence Center "Intelligence"
& Senior Cloud Consultant

Devoteam GmbH, Industriestr. 3, 70565 Stuttgart, Germany
Phone: +49 6151 868-8725, Fax: +49 711 13353-53, Mobile: +49 171 864 1543
E-Mail: juergen.wagner@devoteam.com
<mailto:juergen.wagner@devoteam.com>, URL: www.devoteam.de
<http://www.devoteam.de/>

------------------------------------------------------------------------
Managing Board: Jürgen Hatzipantelis (CEO)
Address of Record: 64331 Weiterstadt, Germany; Commercial Register:
Amtsgericht Darmstadt HRB 6450; Tax Number: DE 172 993 071



Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message