kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamal C <kamaltar...@gmail.com>
Subject Re: Kafka Cluster Issue
Date Mon, 04 May 2015 05:05:44 GMT
We are running ZooKeeper in ensemble (Cluster of 3 / 5).  With further
investigation, I found that the Connect Exception throws for all "inflight"
producers.

Say we are pushing 50 msg/s to a topic. Stop the leader Kafka for that
topic. Producers are unable to push messages to the Kafka Cluster and
starts throwing the Connect Exception.



*Exception* WARN 2015-05-04 10:27:41,052 [kafka-producer-network-thread |
NOTIFICATION_CATEGORY_ALARM]: Selector:poll() :  : Error in I/O with
tcstest2.nmsworks.co.in/192.168.11.140
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[?:1.7.0_40]
        at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
~[?:1.7.0_40]
        at org.apache.kafka.common.network.Selector.poll(Selector.java:238)
[kafka-clients-0.8.2.0.jar:?]
        at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192)
[kafka-clients-0.8.2.0.jar:?]
        at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191)
[kafka-clients-0.8.2.0.jar:?]
        at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122)
[kafka-clients-0.8.2.0.jar:?]


*Description of a Kafka topic*

[root@tcstest2 bin]# sh kafka-topics.sh --zookeeper localhost:2181
--describe | grep NOTIFICATION_CATEGORY_ALARM
Topic:NOTIFICATION_CATEGORY_ALARM    PartitionCount:1
ReplicationFactor:3    Configs:
    Topic: NOTIFICATION_CATEGORY_ALARM    Partition: 0    Leader: 1
Replicas: 2,0,1    Isr: 1,0


Leader is switching but producers are unable to find the new leader. How to
resolve it?


On Sun, May 3, 2015 at 11:13 PM, Jiangjie Qin <jqin@linkedin.com.invalid>
wrote:

> What do you mean by cluster mode with 3 Zookeeper and 3 Kafka brokers? Do
> you mean 1 Zookeeper and 3 brokers?
>
> On 5/2/15, 11:01 PM, "Kamal C" <kamaltarget@gmail.com> wrote:
>
> >Any comments on this issue?
> >
> >On Sat, May 2, 2015 at 9:16 AM, Kamal C <kamaltarget@gmail.com> wrote:
> >
> >> Hi,
> >> We are using Kafka_2.10-0.8.2.0, new Kafka producer and Kafka Simple
> >> Consumer. In Standalone mode, 1 ZooKeeper and 1 Kafka we haven't faced
> >>any
> >> problems.
> >>
> >> In cluster mode, 3 ZooKeeper and 3 Kafka Brokers. We did some sanity
> >> testing by bringing a Kafka node down then a random Producer starts to
> >> throw Connect Exception continuously and tries to connect with the dead
> >> node (not all producers).
> >>
> >> Is there any configuration available to avoid this exception ?
> >>
> >> Regards,
> >> Kamal C
> >>
> >>
> >>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message