kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjkosh...@gmail.com>
Subject Re: Create Kafka Topic Programatically
Date Wed, 20 Jan 2016 17:39:04 GMT
https://issues.apache.org/jira/browse/KAFKA-2945 and related jiras is
probably what you are looking for. That is planned for the next release.

On Wed, Jan 20, 2016 at 8:41 AM, Tommy Becker <tobecker@tivo.com> wrote:

> This works, but it's clumsy and has limitations. Unfortunately, I'm not
> aware of any alternatives. We do some programmatic topic creation when we
> want to create a topic with a non-default configuration. But specifying the
> configuration is an all or nothing affair. For example, there's no way to
> say create this topic using all the defaults except enable log compaction.
> Worse, I don't think there's a way to get what the default topic
> configuration is from the client. I'd like to see some improvements here.
>
> On 01/20/2016 08:24 AM, Timo Ahokas wrote:
>
> Hi Joe,
>
> We're doing a similar thing that you're looking for with some of our app
> nodes. We use the kafka.admin.AdminTools and its
> topicExists()/createTopic() methods. Some additional code (e.g. Kafka
> context check/creation) might be needed for dev/test environments if you're
> using a specific Kafka context instead of the root.
>
> -Timo
>
> On 20 January 2016 at 23:14, Joe San <codeintheopen@gmail.com><mailto:
> codeintheopen@gmail.com> wrote:
>
>
>
> I doubt that might be enough. Could you tell me if the
> auto.create.topics.enable satisfies the following requirement?
>
> 1. I want to create a topic with a specific name
> 2. If I restart the producer client, if the topic with that name already
> exists, it should do nothing and use the topic as is
> 3. Upon producer restart, if the topic already exists, and if the topic has
> un consumed messages, it should not be deleted
>
> Thanks and Regards,
> Joe
>
> On Wed, Jan 20, 2016 at 2:11 PM, Franco Giacosa <fgiacosa@gmail.com
> ><mailto:fgiacosa@gmail.com>
> wrote:
>
>
>
> Hi Joe,
>
> There is an option in the producer called auto.create.topics.enable, so
>
>
> the
>
>
> producer can just start sending data to a topic and the topic will be
> created with the default values.
>
>
>
> 2016-01-20 13:19 GMT+01:00 Joe San <codeintheopen@gmail.com><mailto:
> codeintheopen@gmail.com>:
>
>
>
> Kafka Users,
>
> How can I create a kafka topic programatically?
>
> I would like to create the topics when I initialize my application. It
> should also be in such a way that if the topic already exists, the
> initialization code should do nothing!
>
> Thanks and Regards,
> Joe
>
>
>
>
>
>
>
>
>
>
>
>
> --
> Tommy Becker
> Senior Software Engineer
>
> Digitalsmiths
> A TiVo Company
>
> www.digitalsmiths.com<http://www.digitalsmiths.com>
> tobecker@tivo.com<mailto:tobecker@tivo.com>
>
> ________________________________
>
> This email and any attachments may contain confidential and privileged
> material for the sole use of the intended recipient. Any review, copying,
> or distribution of this email (or any attachments) by others is prohibited.
> If you are not the intended recipient, please contact the sender
> immediately and permanently delete this email and any attachments. No
> employee or agent of TiVo Inc. is authorized to conclude any binding
> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
> Inc. may only be made by a signed written agreement.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message