kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Palino <tpal...@gmail.com>
Subject Re: Frequent Consumer and Producer Disconnects
Date Fri, 25 Sep 2015 20:33:15 GMT
I don't see the logs attached, but what does the GC look like in your
applications? A lot of times this is caused (at least on the consumer side)
by the Zookeeper session expiring due to excessive GC activity, which
causes the consumers to go into a rebalance and change up their connections.

-Todd


On Fri, Sep 25, 2015 at 1:25 PM, Gwen Shapira <gwen@confluent.io> wrote:

> How busy are the clients?
>
> The brokers occasionally close idle connections, this is normal and
> typically not something to worry about.
> However, this shouldn't happen to consumers that are actively reading data.
>
> I'm wondering if the "consumers not making any progress" could be due to a
> different issue, and because they are idle, the connection closes (vs the
> other way around).
>
> On Thu, Sep 24, 2015 at 2:32 PM, noah <iamnoah@gmail.com> wrote:
>
> > We are having issues with producers and consumers frequently fully
> > disconnecting (from both the brokers and ZK) and reconnecting without any
> > apparent cause. On our production systems it can happen anywhere from
> every
> > 10-15 seconds to 15-20 minutes. On our less beefy test systems and
> > developer laptops, it can happen almost constantly.
> >
> > We see no errors in the logs (sample attached), just a message for each
> of
> > our our consumers and producers disconnecting, then reconnecting. On the
> > systems where it happens constantly, the consumers are not making any
> > progress.
> >
> > The logs on the brokers are equally unhelpful, they show only frequent
> > connects and reconnects, without any apparent cause.
> >
> > What could be causing this behavior?
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message