kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chintan mavawala <chintan25...@gmail.com>
Subject Re: Dynamic topic subscription without using pattern
Date Sat, 06 Jan 2018 16:43:21 GMT
In theory pattern "." would work. But that would mean subscribing on all
topics which I don't want.

I share same pain in using pattern which is why I would like to know if it
is good practice to subscribe to topics after kafka consumer has started.

Regards,
Chintan

On 06-Jan-2018 5:47 PM, "Skip Montanaro" <skip.montanaro@gmail.com> wrote:

In theory, wouldn't

consumer.subscribe(pattern=".")

work? I say "in theory" because my experience with subscribing by pattern
hasn't been great. I suspect my mental model of how it's implemented isn't
a close approximation of reality.

Skip

On Jan 6, 2018 4:07 AM, "chintan mavawala" <chintan25487@gmail.com> wrote:

I want to subscribe to all topics as they are created but topic names does
not have any pattern. Consumer starts with subscription to let's say 3
topics and add more topics to exisiting kafka consumer as they are created.

Regards,
Chintan

On 04-Jan-2018 11:02 PM, "Jordan Pilat" <jrpilat@gmail.com> wrote:

> Did you want to subscribe to all topics as they are created, or to exclude
> some of them?
> If the latter, is there a pattern in the names of the topics you wish to
> exclude?
>
> - Jordan Pilat
>
> On 2018-01-03 23:39, chintan mavawala <chintan25487@gmail.com> wrote:
> > Hi,
> >
> > We have a use case where we need to start kafka consumer with a fixed
> list
> > of kafka topics and add more topics on the fly. Since there is no
pattern
> > in names of topics, using pattern for dynamic subscriptions of topics is
> > not feasible.
> >
> > Is it a good practice to subscribe to topics on given kafka consumer on
> fly
> > using after starting consumer?
> >
> > I understand that KafkaConsumer is not thread safe. Any ill effects due
> to
> > multi threaded env?
> >
> > Regards,
> > Chintan
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message