kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Seshadri, Balaji" <Balaji.Sesha...@dish.com>
Subject RE: Controller is not being failed over 0.8.1
Date Thu, 17 Apr 2014 13:54:02 GMT
The ZK data was showing controller as still broker-1.

@Bob,

Can you please send errors you see in the state-change.log.

Thanks,

Balaji
________________________________________
From: Jun Rao [junrao@gmail.com]
Sent: Wednesday, April 16, 2014 9:37 PM
To: users@kafka.apache.org
Subject: Re: Controller is not being failed over 0.8.1

What's the controller value in the zk path (see
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper)?
Any error in the controller/state-change log?

Thanks,

Jun


On Wed, Apr 16, 2014 at 10:07 AM, Seshadri, Balaji <Balaji.Seshadri@dish.com
> wrote:

> Hi,
>
> We got the following error spamming the logs when broker 1 is the
> controller and we are shutting it down in controlled manner not kill -9.
>
> The leader being switched to broker 2 for all partitions but controller is
> not being failed over to broker 2.
>
> [2014-04-16 10:48:47.976-0600] ERROR
> [Controller-1-to-broker-1-send-thread], Controller 1's connection to broker
> id:1,host:tm1-kafkabroker101,port:9092 was unsuccessful
> (kafka.controller.RequestSendThread)
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.Net.connect0(Native Method)
>         at sun.nio.ch.Net.connect(Net.java:465)
>         at sun.nio.ch.Net.connect(Net.java:457)
>         at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>         at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
>         at
> kafka.controller.RequestSendThread.connectToBroker(ControllerChannelManager.scala:173)
>         at
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:140)
>         at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
> [2014-04-16 10:48:48.276-0600] ERROR
> [Controller-1-to-broker-1-send-thread], Controller 1 epoch 38 failed to
> send LeaderAndIsr request with correlation id 1766 to broker
> id:1,host:tm1-kafkabroker101,port:9092. Reconnecting to broker.
> (kafka.controller.RequestSendThread)
> java.nio.channels.ClosedChannelException
>         at kafka.network.BlockingChannel.send(BlockingChannel.scala:89)
>         at
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:132)
>         at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
> [2014-04-16 10:48:48.277-0600] ERROR
> [Controller-1-to-broker-1-send-thread], Controller 1's connection to broker
> id:1,host:tm1-kafkabroker101,port:9092 was unsuccessful
> (kafka.controller.RequestSendThread)
> java.net.ConnectException: Connection refused
>         at sun.nio.ch.Net.connect0(Native Method)
>         at sun.nio.ch.Net.connect(Net.java:465)
>         at sun.nio.ch.Net.connect(Net.java:457)
>         at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)
>         at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
>         at
> kafka.controller.RequestSendThread.connectToBroker(ControllerChannelManager.scala:173)
>         at
> kafka.controller.RequestSendThread.liftedTree1$1(ControllerChannelManager.scala:140)
>         at
> kafka.controller.RequestSendThread.doWork(ControllerChannelManager.scala:131)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
>
>
> It stopped our upgrade because of this issue.
>
> Thanks,
>
> Balaji
>
>
>

Mime
View raw message