kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Dyachkov <andrey.dyach...@gmail.com>
Subject Re: Kafka streams consumer/producer throttling
Date Tue, 20 Nov 2018 08:52:11 GMT
Hi Guozhang,

I set client id like that:
propsWithAppId.put(StreamsConfig.APPLICATION_ID_CONFIG, appId);
propsWithAppId.put(StreamsConfig.CLIENT_ID_CONFIG, appId);
propsWithAppId.put(ProducerConfig.CLIENT_ID_CONFIG, appId);
propsWithAppId.put(ConsumerConfig.CLIENT_ID_CONFIG, appId);
propsWithAppId.put(StreamsConfig.producerPrefix(ProducerConfig.CLIENT_ID_CONFIG),
appId);
propsWithAppId.put(StreamsConfig.consumerPrefix(ConsumerConfig.CLIENT_ID_CONFIG),
appId);

after I tried to set it in 2 different ways neither of them worked:
AdminUtils.changeClientIdConfig(zkUtils, clientId, clientProps); // i
know it is deprecated
 bin/kafka-configs.sh  --zookeeper zk-ips --alter --add-config
'producer_byte_rate=1048576,consumer_byte_rate=1048576' --entity-type
clients --entity-name appId

I checked the source code of the kstreams. The client id is actually
changed by kstreams, that's why setting it does not have any effect.
It forms client id as: clientId + thread name + (consumer | producer |
restore-consumer)

I have not found in the docs if it is possible to use wildcard when
setting client id bandwidth threshold.

On Sat, 17 Nov 2018 at 01:23, Guozhang Wang <wangguoz@gmail.com> wrote:
>
> Hello Andrey,
>
> Could you provide a bit more information on how you set the bandwidth based
> on client id? Sharing some code snippet would even better for me to
> understand your encountered issue.
>
> Guozhang
>
> On Fri, Nov 9, 2018 at 7:11 AM Andrey Dyachkov <andrey.dyachkov@gmail.com>
> wrote:
>
> > Hello,
> >
> > Could you tell me if there is a way to throttle kafka streams by client id?
> > I’ve tried to set bandwidth limit for client id, which is specefied in
> > kafka stream config, but it does not work.
> > --
> >
> > With great enthusiasm,
> > Andrey
> >
>
>
> --
> -- Guozhang



-- 
Thanks,
Andrey

Mime
View raw message