kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kabessa, Itzik" <kabe...@microfocus.com>
Subject Question about Kafka uptime
Date Thu, 11 Oct 2018 13:17:09 GMT
Hi,
We are using a dockerized version of Kafka :
https://github.com/wurstmeister/kafka-docker

We deploy using ECS on AWS machines.
We have "tests" in our system, and for each "test" we have new Kafka and Zookeeper brokers
(for security reasons, it's a must).

We run on EC2, and using Elastic IP(it's a must as well).

We have several topics that are being created when Kafka broker goes up. (using AUTO_CREATE_TOPICS
env. Variable)

Performance is crucial for us, and it's important for us to have all topics ready before we
can start using Kafka.(our use cases does not allow dynamics topics creation)

We see that there is a major difference between the topics creation time between several times
that we start Kafka.
Sometimes, it takes 12 seconds to create the topics, and sometimes it can take 2 minutes.

We would really like to make the topics creation process:

1.      Short

2.      Deterministic

Do you have any suggestions/best practices regarding topics creation?
How can we achieve what we are after?

Thanks,
Great weekend.

Itzik

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message