kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sharninder Khera <sharnin...@gmail.com>
Subject Re: Multiple instances of HL Consumer
Date Fri, 05 Jun 2015 14:28:20 GMT
You can have the same consumer id and Kafka will balance partitions across the two instances
automatically. When one of them dies the partitions are rebalanced and assigned to the remaining
alive consumers. 

From: Panda, Samaresh <samaresh.panda@washpost.com>
Sent: Friday, June 5, 2015 7:42 pm
Subject: Multiple instances of HL Consumer
To:  <users@kafka.apache.org>

I've a HL consumer receiving messages using four threads (four partitions). This is a stand-alone
Java client. For fail-safe reasons, I want to run another instance of the exact same Java
client in a different box.

Here are my questions:

> Can I keep the same consumer group name or it must be different for the 2nd instance?
> If same consumer group, will the 2nd client receive same set of messages again?
> In general what's the best practice to designing fail-safe clients?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message