lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manohar Sripada <manohar...@gmail.com>
Subject Re: Solr node not found in ZK live_nodes
Date Wed, 07 Dec 2016 05:34:01 GMT
Thanks Erick! Should I create a JIRA issue for the same?

Regarding the logs, I have changed the log level to WARN. That may be the
reason, I couldn't get anything from it.

Thanks,
Manohar

On Tue, Dec 6, 2016 at 9:58 PM, Erick Erickson <erickerickson@gmail.com>
wrote:

> Most likely reason is that the Solr node in question,
> was not reachable thus it was removed from
> live_nodes. Perhaps due to temporary network
> glitch, long GC pause or the like. If you're rolling
> your logs over it's quite possible that any illuminating
> messages were lost. The default 4M size for each
> log is quite lo at INFO level...
>
> It does seem possible for a Solr node to periodically
> check its status and re-insert itself into live_nodes,
> go through recovery and all that. So far most of that
> registration logic is baked into startup code. What
> do others think? Worth a JIRA?
>
> Erick
>
> On Tue, Dec 6, 2016 at 3:53 AM, Manohar Sripada <manohar211@gmail.com>
> wrote:
> > We have a 16 node cluster of Solr (5.2.1) and 5 node Zookeeper (3.4.6).
> >
> > All the Solr nodes were registered to Zookeeper (ls /live_nodes) when
> setup
> > was done 3 months back. Suddenly, few days back our search started
> failing
> > because one of the solr node(consider s16) was not seen in Zookeeper,
> i.e.,
> > when we checked for *"ls /live_nodes"*, *s16 *solr node was not found.
> > However, the corresponding Solr process was up and running.
> >
> > To my surprise, I couldn't find any errors or warnings in solr or
> zookeeper
> > logs related to this. I have few questions -
> >
> > 1. Is there any reason why this registration to ZK was lost? I know logs
> > should provide some information, but, it didn't. Did anyone encountered
> > similar issue, if so, what can be the root cause?
> > 2. Shouldn't Solr be clever enough to detect that the registration to ZK
> > was lost (for some reason) and should try to re-register again?
> >
> > PS: The issue is resolved by restarting the Solr node. However, I am
> > curious to know why it happened in the first place.
> >
> > Thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message