kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: Producer throughput with varying acks=0,1,-1
Date Mon, 19 Nov 2018 18:07:14 GMT
The producer has a config `max.in.flight.requests.per.connection` and
because of `ack=all`, even if you don't check the `Future`, a request
will be in-flight longer, because the leader needs to wait for the
followers to replicate the data before it can send the ack back.

Thus, I assume that you are limited by
`max.in.flight.request.per.connection` (you could try to increase it
though). However, if you don't check the `Future` from the producer,
setting ack=all is pointless anyway (and I would consider it a
miss-configuration).


-Matthias

On 11/18/18 8:41 PM, Srinivas, Kaushik (Nokia - IN/Bangalore) wrote:
> But if sends are not done in blocking way (with .get()) how does acks matter ?
> 
> -----Original Message-----
> From: Matthias J. Sax <matthias@confluent.io> 
> Sent: Saturday, November 17, 2018 12:15 AM
> To: users@kafka.apache.org
> Subject: Re: Producer throughput with varying acks=0,1,-1
> 
> I you enable acks, it's not fire and forget any longer.
> 
> -Matthias
> 
> On 11/16/18 1:00 AM, Abhishek Choudhary wrote:
>> Hi,
>>
>> I have been doing some performance tests with kafka cluster for my project.
>> I have a question regarding the send call and the 'acks' property of 
>> producer. I observed below numbers with below invocation of send call. 
>> This is a simple fire and forget call.
>>
>> producer.send(record);
>>
>> The topic has 5 partitions and I see below results with different acks 
>> value and replication factor. The kafka cluster has 5 nodes running 
>> with default values and using local disk
>>
>> *acks                    Replication factor=1
>> Replication factor=3*
>> 0                             1330k msgs/sec
>>      1260k msgs/sec
>> 1                             1220k msgs/sec
>>      1200k msgs/sec
>> -1(all)                      1220k msgs/sec
>>     325k msgs/sec
>>
>>
>> As you can see as the acks value changes from 0 to all, the producer 
>> throughput decreases. What I am not able to understand is that if the 
>> producer send call is fire and forget in nature and producer is not 
>> waiting for any acknowledgements then why does the producer throughput 
>> drops as we move to stronger acks guarantees?
>>
>> Any insights into how acks and producer send call works in Kakfa would 
>> be greatly appreciated.
>>
>> Thanks,
>> Abhishek
>>
> 


Mime
View raw message