kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ricardo Ferreira <rifer...@riferrei.com>
Subject Re: handle multiple requests
Date Wed, 10 Jun 2020 17:04:23 GMT
Hi there,

Unless you are dealing with a low volume scenario, you should avoid tie 
each message/record to a specific thread. It will limit your ability to 
scale the processing out as CPU is a scarce resource. Alternatively, you 
should write your code to fetch multiple records at once (like a batch) 
and process them using the same thread that performed the fetch -- or 
hand over to another thread to decouple consumption from processing. 
Regardless, each thread should handle way more than just one record at a 
time. You can still control how each record is deemed processed by 
handling each offset individually.

This link <https://docs.confluent.io/current/clients/java.html#ak-java> 
contains a good intro about Kafka's consumer API.

Thanks,

-- Ricardo

On 6/10/20 10:20 AM, נתי אלמגור wrote:
> hello
> i'm very new in Kafka , i have experience in Rabbit MQ
> i have connection Layer which publish to Rabbit queue request and worker app which have
number of threads(for example 8 equal to CPU's number) that subscribe to Rabbit Queue and
each request handled with one thread
>
> i can not  find  this solution in kafka  please help me

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message