kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reza Aliakbari <raliakb...@gmail.com>
Subject Partition Consumer(s)
Date Thu, 10 Sep 2015 14:54:39 GMT
Hi Everybody,

I have 2 question regarding the way consumers, consume messages of a

   - * Is it possible to configure Kafka to allow concurrent message
   consumption from one partition concurrently? The order is not my concern at

           I couldn't find any way to that by the Group Of Consumer
approach, If it is possible please let me know, If impossible, then let me
know how to address this problem:
           For a reason a consumer that is assigned to a partition could
get very slow, and the messages would be processed very slowly. How can I
detect this and stop producing on this slow partition...

   - * Suppose I have 5 partitions and 3 consumers and I am using Group of
   Consumers model(I had 5 consumers at start but 2 servers crashed), 3
   consumers are working busy with their 3   partitions and they never get
   finished since the producer produce to their partitions **non-stop and a
   little faster than their consumption. What happens to the other 2
   partitions that are missing consumers? How the Group of Consumers can
   handle this issue?*

*The order is no matter for me, I need a simple configuration that address
my concurrency needs and I need to make sure no message gets into
starvation scenario that never consumed.*

Please let us know, we want to select between Kafka and RabitMQ and we
prefer Kafka because it is growing community and high throughput, But first
we need to address these basic needs.


Reza Aliakabri

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message