kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@apache.org>
Subject Re: Kafka Streams : Rack Aware deployment in EC2
Date Mon, 19 Oct 2020 18:26:00 GMT
No suggestions... Sorry.

The task assignment algorithm cannot be customized.

The only thing you _could_ do, is to pickup the ticket yourself (to get
the feature maybe in 2.8 release). Not sure if you would be interested
to contribute :)


-Matthias

On 10/19/20 11:08 AM, Tony John wrote:
> Thanks for the quick response Matthias. We were planning to move to AWS
> MSK, and I think we will have to defer it until rack awareness is in place
> for Kafka Streams. Reason being consumers consuming from brokers in
> different availability zones would result in high data transfer charges. Do
> you have any suggestions?
> 
> Thanks,
> Tony
> 
> On Mon, Oct 19, 2020 at 11:08 PM Matthias J. Sax <mjsax@apache.org> wrote:
> 
>> There is no such feature for Kafka Streams yet.
>>
>> We have one ticket for rack aware standby task placement
>> (https://issues.apache.org/jira/browse/KAFKA-6718) but nobody is working
>> on it.
>>
>>
>> -Matthias
>>
>> On 10/19/20 9:20 AM, Tony John wrote:
>>> Hi All,
>>>
>>> I have been trying to figure out some documentation on how to enable rack
>>> awareness on a Kafka Streams app. I do see the broker.rack configuration
>>> which needs to be done at the broker side and client.rack configuration
>> for
>>> the consumers. Is there any specific configuration which is required to
>>> enable rack awareness in a streams app? Please throw some light on this.
>>>
>>> PS: I am using Kafka / Kafka Streams 2.2.1
>>>
>>> Thanks,
>>> Tony
>>>
>>
> 

Mime
View raw message