kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kamal C <kamaltar...@gmail.com>
Subject Re: Kafka consumers in cluster
Date Thu, 04 Aug 2016 06:00:55 GMT
Implement your own custom
and assign all the subscribed partitions to the first consumer instance in
the group.

See 'partition.assignment.strategy' config in the consumer configs [1]

[1]: http://kafka.apache.org/documentation.html#newconsumerconfigs

On Thu, Aug 4, 2016 at 8:54 AM, sat <sathish.alwar@gmail.com> wrote:

> Hi,
> We have Kafka server/broker running in a seperate machine (say machine A),
> for now we are planning to have in one node. We have multiple topics and
> all topics have only 1 partition for now.
> We have our application which includes Kafka consumers installed in machine
> B and machine C. Our application in machine B and C are in cluster, hence
> Kafka Consumers will also be in cluster. Both our consumers will have same
> group id. We want all the messages to be consumed by consumer in machine B
> and only when machine B is down consumer in machine C should pull messages.
> Since consumer in machine B and C have same group id, we came to know
> consumer in machine B will get message for some time duration (10mins) and
> then consumer in machine C will get message for some time duration. Since
> our consumers are in cluster, we want only consumer to be active or receive
> all the messages as long as it is alive.
> Please let us know how to achieve this.
> Thanks and Regards
> A.SathishKumar
> 044-24735023

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message