kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ali Akhtar <ali.rac...@gmail.com>
Subject Re: Adding topics to KafkaStreams after ingestion has been started?
Date Fri, 02 Dec 2016 14:23:28 GMT
That's pretty useful to know - thanks.

1) If I listened too foo-.*, and there were 5 foo topics created after
kafka streaming was running: foo1, foo2, foo3, foo4, foo5, will this create
5 consumers / threads / instances, or will it be just 1 instance that
receives the messages for all of those topics?

2) Will this cause issues performance issues if i had a lot of throwaway
foo topics being created, or will this scale?

On Fri, Dec 2, 2016 at 7:17 PM, Damian Guy <damian.guy@gmail.com> wrote:

> Hi Ali,
>
> The only way KafkaStreams will process new topics after start is if the
> original stream was defined with a regular expression, i.e,
> kafka.stream(Pattern.compile("foo-.*");
>
> If any new topics are added after start that match the pattern, then they
> will also be consumed.
>
> Thanks,
> Damian
>
> On Fri, 2 Dec 2016 at 13:13 Ali Akhtar <ali.rac200@gmail.com> wrote:
>
> > Heya,
> >
> > Normally, you add your topics and their callbacks to a StreamBuilder, and
> > then call KafkaStreams.start() to start ingesting those topics.
> >
> > Is it possible to add a new topic to the StreamBuilder, and start
> ingesting
> > that as well, after KafkaStreams.start() has been called?
> >
> > Thanks.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message