kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andras Hatvani <andras.hatv...@andrashatvani.com>
Subject Re: LeaderNotAvailableException, although leader elected
Date Fri, 26 Sep 2014 14:07:58 GMT
And the solution was:
Increase retry.backoff.ms from the default 100 ms to 1000 ms, so the output is:

11891 [main] INFO  kafka.client.ClientUtils$ - Fetching metadata from broker id:0,host:192.168.59.103,port:9092
with correlation id 0 for 1 topic(s) Set(inputTopic)
11893 [main] INFO  kafka.producer.SyncProducer - Connected to 192.168.59.103:9092 for producing
12045 [main] INFO  kafka.producer.SyncProducer - Disconnecting from 192.168.59.103:9092
12062 [main] WARN  kafka.producer.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata
for topic inputTopic -> 
No partition metadata for topic inputTopic due to kafka.common.LeaderNotAvailableException}]
for topic [inputTopic]: class kafka.common.LeaderNotAvailableException 
12066 [main] INFO  kafka.client.ClientUtils$ - Fetching metadata from broker id:0,host:192.168.59.103,port:9092
with correlation id 1 for 1 topic(s) Set(inputTopic)
12067 [main] INFO  kafka.producer.SyncProducer - Connected to 192.168.59.103:9092 for producing
12097 [main] INFO  kafka.producer.SyncProducer - Disconnecting from 192.168.59.103:9092
12098 [main] WARN  kafka.producer.BrokerPartitionInfo - Error while fetching metadata [{TopicMetadata
for topic inputTopic -> 
No partition metadata for topic inputTopic due to kafka.common.LeaderNotAvailableException}]
for topic [inputTopic]: class kafka.common.LeaderNotAvailableException 
12098 [main] ERROR kafka.producer.async.DefaultEventHandler - Failed to collate messages by
topic, partition due to: Failed to fetch topic metadata for topic: inputTopic
12099 [main] INFO  kafka.producer.async.DefaultEventHandler - Back off for 1000 ms before
retrying send. Remaining retries = 3
13104 [main] INFO  kafka.client.ClientUtils$ - Fetching metadata from broker id:0,host:192.168.59.103,port:9092
with correlation id 2 for 1 topic(s) Set(inputTopic)
13111 [main] INFO  kafka.producer.SyncProducer - Connected to 192.168.59.103:9092 for producing
13137 [main] INFO  kafka.producer.SyncProducer - Disconnecting from 192.168.59.103:9092
13161 [main] INFO  kafka.producer.SyncProducer - Connected to 192.168.59.103:9092 for producing

In case this would be not enough for you, you can try to change the values of 
- message.send.max.retries from the default 5 to e.g. 10 and
- topic.metadata.refresh.interval.ms to 0


It is still unclear, why it takes 2 tries i.e. 2 seconds to fetch the topic metadata…


Cheers,
Andras

> On 26 Sep 2014, at 14:43, Andras Hatvani <andras.hatvani@andrashatvani.com> wrote:
> 
> Hi,
> 
> What does the exception mean in the following context?
> 
> I've got two Kafka brokers out which the second one will be elected as leader:
> 
> kafka2_1          | [2014-09-26 12:35:07,289] INFO [Kafka Server 2], started (kafka.server.KafkaServer)
> kafka2_1          | [2014-09-26 12:35:07,394] INFO New leader is 2 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
> 
> Then, when I try to produce a message, I only get the following error:
> 
> 23182 [main] INFO  kafka.producer.async.DefaultEventHandler - Back off for 100 ms before
retrying send. Remaining retries = 0
> 23286 [main] INFO  kafka.client.ClientUtils$ - Fetching metadata from broker id:0,host:192.168.59.103,port:9092
with correlation id 8 for 1 topic(s) Set(inputTopic)
> 23287 [main] INFO  kafka.producer.SyncProducer - Connected to 192.168.59.103:9092 for
producing
> 23300 [main] INFO  kafka.producer.SyncProducer - Disconnecting from 192.168.59.103:9092
> 23301 [main] WARN  kafka.producer.BrokerPartitionInfo - Error while fetching metadata
[{TopicMetadata for topic inputTopic -> 
> No partition metadata for topic inputTopic due to kafka.common.LeaderNotAvailableException}]
for topic [inputTopic]: class kafka.common.LeaderNotAvailableException 
> 23303 [main] ERROR kafka.producer.async.DefaultEventHandler - Failed to send requests
for topics inputTopic with correlation ids in [0,8]
> 
> The property metadata.broker.list contains the host:port values of both brokers.
> 
> What can be wrong/missing?
> Thanks,
> Andras


Mime
View raw message