kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sat <sathish.al...@gmail.com>
Subject Re: Kafka consumers in cluster
Date Thu, 04 Aug 2016 17:54:49 GMT
Hi Kamal,

Thanks for your prompt response. Does our custom partition assignor gets
called during every rebalancing.

Thanks and Regards
A.SathishKumar



>Implement your own custom
>`org.apache.kafka.clients.consumer.internals.PartitionAssignor`
>and assign all the subscribed partitions to the first consumer instance in
>the group.

>See 'partition.assignment.strategy' config in the consumer configs [1]

>[1]: http://kafka.apache.org/documentation.html#newconsumerconfigs



>On Thu, Aug 4, 2016 at 8:54 AM, sat <sathish.alwar@gmail.com> wrote:

> Hi,
>
> We have Kafka server/broker running in a seperate machine (say machine A),
> for now we are planning to have in one node. We have multiple topics and
> all topics have only 1 partition for now.
>
> We have our application which includes Kafka consumers installed in machine
> B and machine C. Our application in machine B and C are in cluster, hence
> Kafka Consumers will also be in cluster. Both our consumers will have same
> group id. We want all the messages to be consumed by consumer in machine B
> and only when machine B is down consumer in machine C should pull messages.
>
> Since consumer in machine B and C have same group id, we came to know
> consumer in machine B will get message for some time duration (10mins) and
> then consumer in machine C will get message for some time duration. Since
> our consumers are in cluster, we want only consumer to be active or receive
> all the messages as long as it is alive.
>
> Please let us know how to achieve this.
>
>
> Thanks and Regards
> A.SathishKumar
> 044-24735023
>



On Wed, Aug 3, 2016 at 8:24 PM, sat <sathish.alwar@gmail.com> wrote:

> Hi,
>
> We have Kafka server/broker running in a seperate machine (say machine A),
> for now we are planning to have in one node. We have multiple topics and
> all topics have only 1 partition for now.
>
> We have our application which includes Kafka consumers installed in
> machine B and machine C. Our application in machine B and C are in cluster,
> hence Kafka Consumers will also be in cluster. Both our consumers will have
> same group id. We want all the messages to be consumed by consumer in
> machine B and only when machine B is down consumer in machine C should pull
> messages.
>
> Since consumer in machine B and C have same group id, we came to know
> consumer in machine B will get message for some time duration (10mins) and
> then consumer in machine C will get message for some time duration. Since
> our consumers are in cluster, we want only consumer to be active or receive
> all the messages as long as it is alive.
>
> Please let us know how to achieve this.
>
>
> Thanks and Regards
> A.SathishKumar
> 044-24735023
>



-- 
A.SathishKumar
044-24735023

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message