kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Håkon Åmdal <hakon.amd...@gmail.com>
Subject Adding or removing input topics to a Kafka Consumer without downtime
Date Tue, 05 Sep 2017 14:00:14 GMT
Hi there,

In our company, we are running multiple Kafka Streams app that are deployed
in a red/black fashion. A deployment means starting a new set of hosts, run
them in parallel with the old hosts until they pass the health check, and
then scale down the old hosts. This approach works very well as we can
deploy application changes without any downtime.

However, as far as I understand, there is no way we can run a new and old
version in parallel if they don’t consume from the same set of topics.
Consumers will try to rebalance tasks between the two different application
version, causing errors like the one below:

Uncaught exception: Thread xyz-StreamThread-1 stopped unexpectedly after
Assigned partition foo-1 for non-subscribed topic regex pattern;
subscription pattern is bar

To mitigate the problem, we have so far scaled down the old cluster to 0
instances before deploying the new application if we’re adding or removing
input topics. However, this causes service downtime which we no longer can
accept.

I’m curious to hear if anyone are experiencing the same issues, or if
anyone have any thoughts or opinions? Are we doing something wrong, or is
this something that can be solved by the Kafka Consumer client?

Thanks,

Håkon

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message