kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommy Becker <tobec...@tivo.com>
Subject Re: Create Kafka Topic Programatically
Date Wed, 20 Jan 2016 16:41:05 GMT
This works, but it's clumsy and has limitations. Unfortunately, I'm not aware of any alternatives.
We do some programmatic topic creation when we want to create a topic with a non-default configuration.
But specifying the configuration is an all or nothing affair. For example, there's no way
to say create this topic using all the defaults except enable log compaction. Worse, I don't
think there's a way to get what the default topic configuration is from the client. I'd like
to see some improvements here.

On 01/20/2016 08:24 AM, Timo Ahokas wrote:

Hi Joe,

We're doing a similar thing that you're looking for with some of our app
nodes. We use the kafka.admin.AdminTools and its
topicExists()/createTopic() methods. Some additional code (e.g. Kafka
context check/creation) might be needed for dev/test environments if you're
using a specific Kafka context instead of the root.


On 20 January 2016 at 23:14, Joe San <codeintheopen@gmail.com><mailto:codeintheopen@gmail.com>

I doubt that might be enough. Could you tell me if the
auto.create.topics.enable satisfies the following requirement?

1. I want to create a topic with a specific name
2. If I restart the producer client, if the topic with that name already
exists, it should do nothing and use the topic as is
3. Upon producer restart, if the topic already exists, and if the topic has
un consumed messages, it should not be deleted

Thanks and Regards,

On Wed, Jan 20, 2016 at 2:11 PM, Franco Giacosa <fgiacosa@gmail.com><mailto:fgiacosa@gmail.com>

Hi Joe,

There is an option in the producer called auto.create.topics.enable, so


producer can just start sending data to a topic and the topic will be
created with the default values.

2016-01-20 13:19 GMT+01:00 Joe San <codeintheopen@gmail.com><mailto:codeintheopen@gmail.com>:

Kafka Users,

How can I create a kafka topic programatically?

I would like to create the topics when I initialize my application. It
should also be in such a way that if the topic already exists, the
initialization code should do nothing!

Thanks and Regards,

Tommy Becker
Senior Software Engineer

A TiVo Company



This email and any attachments may contain confidential and privileged material for the sole
use of the intended recipient. Any review, copying, or distribution of this email (or any
attachments) by others is prohibited. If you are not the intended recipient, please contact
the sender immediately and permanently delete this email and any attachments. No employee
or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc.
by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message