kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AJ Chen <ajc...@web2express.org>
Subject Re: dynamic produce and consume new topics
Date Fri, 12 Jun 2020 19:55:50 GMT
Thanks, Boyang.

Spring''s KafkaTemplate can easily create new topic on the flight already.


On consumer side, is there java sample code to replace this annotation?
That will create a new listener for a new topic and group-id on the
fliight.
 @KafkaListener(topics = "test", groupId = "grp0")

-aj

On Fri, Jun 12, 2020 at 9:30 AM Boyang Chen <reluctanthero104@gmail.com>
wrote:

> Hey AJ,
>
> you should be able to make the subscribed topics and output topics dynamic
> configs for Consumer and Producer. If you need to create new topics on
> runtime, there is some broker side setting you could use to allow topic
> creation based of produced records:
>
> https://stackoverflow.com/questions/36441768/how-to-create-topics-in-apache-kafka
>
> On Fri, Jun 12, 2020 at 9:01 AM AJ Chen <canovaj@gmail.com> wrote:
>
> > I'm working on a data streaming system where users will publish to
> > different new topics and subscribe to different new topics without
> > restarting the system. A large number of topics will be created over
> time.
> > Uer will choose different new topics to publish and subscribe dynamically
> > at runtime. Can someone share good design or practice to implement such
> > dynamic topic handling? Links to sample code will be very helpful.
> >
> > Thanks,
> > aj
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message