lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Scheffler <thomas.scheff...@uni-jena.de>
Subject Re: Memory Leak in 7.3 to 7.4
Date Thu, 02 Aug 2018 11:20:10 GMT
Hi,

SOLR is shipping with a script that handles OOM errors. And produces log files for every case
with content like this:

Running OOM killer script for process 9015 for Solr on port 28080
Killed process 9015

This script works ;-)

kind regards

Thomas



> Am 02.08.2018 um 12:28 schrieb Vincenzo D'Amore <v.damore@gmail.com>:
> 
> Not clear if you had experienced an OOM error.
> 
> On Thu, Aug 2, 2018 at 12:06 PM Thomas Scheffler <
> thomas.scheffler@uni-jena.de> wrote:
> 
>> Hi,
>> 
>> we noticed a memory leak in a rather small setup. 40.000 metadata
>> documents with nearly as much files that have „literal.*“ fields with it.
>> While 7.2.1 has brought some tika issues (due to a beta version) the real
>> problems started to appear with version 7.3.0 which are currently
>> unresolved in 7.4.0. Memory consumption is out-of-roof. Where previously
>> 512MB heap was enough, now 6G aren’t enough to index all files.
>> I am now to a point where I can track this down to the libraries in
>> solr-7.4.0/contrib/extraction/lib/. If I replace them all by the libraries
>> shipped with 7.2.1 the problem disappears. As most files are PDF documents
>> I tried updating pdfbox to 2.0.11 and tika to 1.18 with no solution to the
>> problem. I will next try to downgrade these single libraries back to 2.0.6
>> and 1.16 to see if these are the source of the memory leak.
>> 
>> In the mean time I would like to know if anybody else experienced the same
>> problems?
>> 
>> kind regards,
>> 
>> Thomas
>> 
> 
> 
> --
> Vincenzo D'Amore



Mime
View raw message