kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamesh <kam.iit...@gmail.com>
Subject (Re-)joining group for a longer time
Date Sat, 03 Aug 2019 08:39:51 GMT
Hi,
 I am using Kafka connect cluster for writing data to S3. I have observed,
whenever I add a new connector or update the config of an existing
connector, I think group balancing is happening and it is affecting all the
existing connectors. Rebalancing is happening for all of the existing
connectors also. All of my log files are filled with the following log
messages

*[2019-08-03 08:28:28,668] INFO [Consumer clientId=connector-consumer-xxx,
groupId=connect-xxx] (Re-)joining group
(org.apache.kafka.clients.consumer.internals.AbstractCoordinator:505)*

 This rebalancing is taking a very long time, and sometimes not evening
completing. Any pointers on this?

I am using Kafka connect *2.3.0, *and this version supports incremental
rebalancing and it should not affect existing connectors. Am I
missing something?

I have already tuned *session.timeout.ms <http://session.timeout.ms>*
and *max.poll.interval.ms
<http://max.poll.interval.ms>* configs and increased their values as
suggested in the community.


Thanks & Regards
Kamesh.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message