kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Broker keeps rebalancing
Date Fri, 14 Nov 2014 01:25:26 GMT
Which version of ZK are you using?

Thanks,

Jun

On Thu, Nov 13, 2014 at 10:15 AM, Chen Wang <chen.apache.solr@gmail.com>
wrote:

> Thanks for the info.
> It makes sense, however, I didn't see any "session timeout"/"expired"
> entries in consumer log..
> but do see lots of connection closed entry in zookeeper log:
>
> 2014-11-13 10:07:53,132 [myid:1] - INFO  [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2182:NIOServerCnxn@1007] - Closed socket connection for
> client /10.93.83.50:37180 which had sessionid 0x149a4cc1b580e7d
> 2014-11-13 10:08:04,499 [myid:1] - INFO  [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2182:NIOServerCnxnFactory@197] - Accepted socket
> connection
> from /10.93.80.121:38437
> 2014-11-13 10:08:04,503 [myid:1] - WARN  [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2182:ZooKeeperServer@822] - Connection request from old
> client /10.93.80.121:38437; will be dropped if server is in r-o mode
> 2014-11-13 10:08:04,503 [myid:1] - INFO  [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2182:ZooKeeperServer@868] - Client attempting to establish
> new session at /10.93.80.121:38437
> 2014-11-13 10:08:04,538 [myid:1] - INFO
>  [CommitProcessor:1:ZooKeeperServer@617] - Established session
> 0x149a4cc1b580e7e with negotiated timeout 40000 for client /
> 10.93.80.121:38437
> 2014-11-13 10:08:08,746 [myid:1] - INFO  [NIOServerCxn.Factory:
> 0.0.0.0/0.0.0.0:2182:NIOServerCnxn@1007] - Closed socket connection for
> client /10.93.80.121:38437 which had sessionid 0x149a4cc1b580e7e
>
> We are using -Xmx2048m for consumer, and I didn't see any GC related
> exceptions
>
> Chen
>
>
>
> On Thu, Nov 13, 2014 at 9:13 AM, Guozhang Wang <wangguoz@gmail.com> wrote:
>
> > Hey Chen,
> >
> > As Neha suggested, typical reason of too many rebalances is that your
> > consumers kept being timed out from ZK, and you can verify this by
> checking
> > in your consumer logs for sth. like "session timeout" entries (these are
> > not ERROR entries).
> >
> > Guozhang
> >
> > Guozhang
> >
> > On Wed, Nov 12, 2014 at 5:31 PM, Neha Narkhede <neha.narkhede@gmail.com>
> > wrote:
> >
> > > Does this help?
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog
> > > ?
> > >
> > > On Wed, Nov 12, 2014 at 3:53 PM, Chen Wang <chen.apache.solr@gmail.com
> >
> > > wrote:
> > >
> > > > Hi there,
> > > > My kafka client is reading a 3 partition topic from kafka with 3
> > threads
> > > > distributed on different machines. I am seeing frequent owner changes
> > on
> > > > the topics when running:
> > > > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group
> > > > my_test_group --topic mytopic -zkconnect localhost:2181
> > > >
> > > > The owner kept changing once a while, but I didn't see any exceptions
> > > > thrown from the consumer side. When checking broker log, its full of
> > > >  INFO Closing socket connection to /IP. (kafka.network.Processor)
> > > >
> > > > Is this expected behavior? If so,  how can I tell when  the leader is
> > > > imbalanced, and rebalance is triggered?
> > > > Thanks,
> > > > Chen
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message