kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Soheil Pourbafrani <soheil.i...@gmail.com>
Subject communication between brokers doesn't establish
Date Sat, 05 Jan 2019 12:51:49 GMT
Hi, using Kafka 11.0.3 I set up a 2 node Kafka cluster on CentOS 7 with
broker id 100 and 101. There is no firewall enabled. Both nodes can ping
each other and establish the SSH connection successfully.
When I start the cluster both broker successfully register their id into
the zookeeper cluster (I checked the path /brokers/ids) and the command to
list the topics can be run successfully.

But monitoring brokers logs I see it continuously logs in the file
server.log on the broker with id 100 :
INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
WARN Connection to node 101 could not be established. Broker may not be
available. (org.apache.kafka.clients.Net$
INFO Creating /controller (is it secure? false)
(kafka.utils.ZKCheckedEphemeral)

Also in the controller.log file on the broker id 100, it logs continuously:
INFO [Controller 100]: Broker 100 resigned as the controller
(kafka.controller.KafkaController)
DEBUG [Controller 100]: Controller resigning, broker id 100
(kafka.controller.KafkaController)
DEBUG [Controller 100]: De-registering IsrChangeNotificationListener
(kafka.controller.KafkaController)
INFO [Partition state machine on Controller 100]: Stopped partition state
machine (kafka.controller.PartitionStateMachine)
INFO [Replica state machine on controller 100]: Stopped replica state
machine (kafka.controller.ReplicaStateMachine)
INFO [Controller 100]: Broker 100 resigned as the controller
(kafka.controller.KafkaController)
INFO [Controller 100]: 100 successfully elected as the controller
(kafka.controller.KafkaController)
 INFO [Controller 100]: Broker 100 starting become controller state
transition (kafka.controller.KafkaController)
INFO [Controller 100]: Initialized controller epoch to 96100 and zk version
96099 (kafka.controller.KafkaController)
INFO [Controller 100]: Controller 100 incremented epoch to 96101
(kafka.controller.KafkaController)
DEBUG [Controller 100]: Registering IsrChangeNotificationListener
(kafka.controller.KafkaController)
DEBUG [Channel manager on controller 100]: Controller 100 trying to connect
to broker 100
DEBUG [Channel manager on controller 100]: Controller 100 trying to connect
to broker 101

I also check the network packets using tcpdump and I observed
Zookeeper nodes could get packets from both brokers successfully, but there
isn't any packet has been sent from each broker to the other.

I checked the established connections on both brokers using netstat and
I observed there are much TIME_WAIT established connections between two
brokers.

What could be the reason?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message