kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Lublinsky <boris.lublin...@lightbend.com>
Subject Re: Kafka Streams question
Date Mon, 13 Nov 2017 21:11:30 GMT
I was thinking about controlled stream use case, where one stream is data for processing, while
the second one controls execution.
If I want to scale this, I want to run multiple instances. In this case I want these instances
to share data topic, but control topic should be delivered to all
Instances.
This means that I would like to control group IDs for streams individually

Boris Lublinsky
FDP Architect
boris.lublinsky@lightbend.com
https://www.lightbend.com/

> On Nov 13, 2017, at 2:47 PM, Guozhang Wang <wangguoz@gmail.com> wrote:
> 
> Boris,
> 
> What's your use case scenarios that you'd prefer to set different
> subscriber IDs for different streams?
> 
> 
> Guozhang
> 
> 
> On Mon, Nov 13, 2017 at 6:49 AM, Boris Lublinsky <
> boris.lublinsky@lightbend.com> wrote:
> 
>> This seems like a very limiting implementation
>> 
>> 
>> Boris Lublinsky
>> FDP Architect
>> boris.lublinsky@lightbend.com
>> https://www.lightbend.com/
>> 
>>> On Nov 13, 2017, at 4:21 AM, Damian Guy <damian.guy@gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> The configurations apply to all streams consumed within the same streams
>>> application. There is no way of overriding it per input stream.
>>> 
>>> Thanks,
>>> Damian
>>> 
>>> On Mon, 13 Nov 2017 at 04:49 Boris Lublinsky <
>> boris.lublinsky@lightbend.com>
>>> wrote:
>>> 
>>>> I am writing Kafka Streams implementation (1.0.0), for which I have 2
>>>> input streams.
>>>> Is it possible to have different subscriber IDs for these 2 streams.
>>>> I see only one place where subscriber’s ID can be specified:
>>>> 
>>>> streamsConfiguration.put(StreamsConfig.CLIENT_ID_CONFIG,
>>>> ApplicationKafkaParameters.DATA_GROUP);
>>>> And it does not seem like either Topology or DSL APIs allow to overwrite
>>>> it during Stream creation.
>>>> 
>>>> Thanks for the help
>>>> 
>>>> Boris Lublinsky
>>>> FDP Architect
>>>> boris.lublinsky@lightbend.com
>>>> https://www.lightbend.com/
>>>> 
>>>> 
>> 
>> 
> 
> 
> -- 
> -- Guozhang


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message