lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: 9000+ CLOSE_WAIT connections in solr v6.2.2 causing it to "die"
Date Fri, 02 Feb 2018 22:06:32 GMT
On 2/2/2018 10:00 AM, mmb1234 wrote:
> In our solr non-cloud env., we are seeing lots of CLOSE_WAIT, causing jvm to
> stop "working" with 3 mins of solr start.
>
> solr [ /opt/solr ]$ netstat -anp | grep 8983 | grep CLOSE_WAIT | grep
> 10.xxx.xxx.xxx | wc -l
> 9453

Solr isn't handling network services.  That's being done by the servlet 
container.  The servlet container included with Solr is Jetty.  Since 
version 5.0, Jetty is the only option with official support.

There are a number of issues against Jetty where people are seeing many 
CLOSE_WAIT problems.  At least one of them does acknowledge that there 
is a bug in Jetty, but in discussion with them on IRC, they have 
apparently never been able to actually reproduce the problem.  One issue 
says that they don't think Jetty 9.4.x will have the problem, but Solr 
won't have Jetty 9.4.x until Solr 7.3.0 is released.

You said that you're running Solr 6.2.2, but there is no 6.2.2 version.  
There is 6.4.2 and 6.6.2.  Solr 6.6.x and 6.4.x include Jetty 9.3.14, 
6.2.x includes Jetty 9.3.8.

Another oddity: Your info says that Solr has a 30GB heap, but the JVM 
argument list includes "-Xmx512m" which is a 512MB heap.

One issue that a many open connections causes is a large number of open 
file handles, and that can apparently cause software to become 
unresponsive.  Are you seeing any errors in Solr's logfile before the 
process becomes unresponsive?

Thanks,
Shawn


Mime
View raw message