kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stevo Slavić <ssla...@gmail.com>
Subject Explicit topic creation and topic metadata availability
Date Wed, 22 Oct 2014 12:57:23 GMT
Hello Apache Kafka users,

Using Kafka 0.8.1.1 (single instance with single ZK 3.4.6 running locally),
with auto topic creation disabled, in a test I have topic created with
AdminUtils.createTopic (AdminUtils.topicExists returns true) but
KafkaProducer on send request keeps throwing
UnknownTopicOrPartitionException even after 100 retries, both when
topic.metadata.refresh.interval.ms and retry.backoff.ms are left at
defaults, and when customized.

Am I doing something wrong or is this a known bug?

How long does it typically take for metadata to be refreshed?
How long does it take for leader to be elected?

Documentation for retry.backoff.ms states:
"Before each retry, the producer refreshes the metadata of relevant topics
to see if a new leader has been elected. Since leader election takes a bit
of time, this property specifies the amount of time that the producer waits
before refreshing the metadata."

Do I understand this docs correctly - on failure to send a message, such as
unknown topic, if retries are configured producer will wait for configured
retry.backoff.ms, then it will initiate and wait for metadata refresh to
complete, and only then retry sending?

Kind regards,
Stevo Slavic.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message