kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: when will the messsages be sent to broker
Date Thu, 02 Mar 2017 05:11:19 GMT
That's weird. If batch.size=0 and linger.ms=0 records should get send
out immediately.

buffer.memory is an upper bound for the overall memory. If you write to
multiple partitions, there will be a buffer for each partitions and
buffer.memory is shared over all buffers. Furthermore, if you exceed
buffer.memory your call to Producer#send will block (until it might time
out). I don't think you want to set it to zero.

How do you monitor your Producer to come to the conclusion it's not
sending immediately?

-Matthias


On 3/1/17 6:30 PM, Yuanjia wrote:
> Hi,
> Records are not sent out  immediately, even if configing batch.size=0 and linger.ms=0,
They are buffered in the memory(buffer.memory).
> I don't want to do sync writes by future.get, which performance is not good.
> Have any configure can increase sending frequency? config buffer.memory=0?
> 
> Thanks.
> 
> Yuanjia Li
>  
> From: Matthias J. Sax
> Date: 2017-03-02 01:42
> To: users
> Subject: Re: when will the messsages be sent to broker
> There is also linger.ms parameter that is an upper bound how long a (not
> yet filled) buffer is hold before sending it even if it's not full.
>  
> Furthermore, you can do sync writes and block until producer received
> all acks. But it might have a performance penalty.
>  
> http://docs.confluent.io/current/clients/producer.html#synchronous-writes
>  
>  
>  
> -Matthias
>  
> On 3/1/17 6:55 AM, Kaufman Ng wrote:
>> By default producer batches records (into a buffer) until batch.size is
>> reached, then records will be sent out.  Try lowering batch.size in your
>> producer config.
>>
>> You might want to look at the producer javadoc here:
>> http://kafka.apache.org/0102/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html
>>
>> On Wed, Mar 1, 2017 at 3:14 AM, Yuanjia <yuanjia8947@163.com> wrote:
>>
>>> Hi all,
>>> When will the messsages be sent in kafka0.10.0?If I use KafkaProducer.send
>>> to send one message, the messsages isn't sent immediately except invoke
>>> flush or close.
>>>
>>> Thanks.
>>>
>>>
>>>
>>> Yuanjia Li
>>>
>>
>>
>>
>  
> 【来自网易邮箱的超大附件】
> 邮件带有附件预览链接,若您转发或回复此邮件时不希望对方预览附件,建议您手动删除链接。
>  
> signature.asc
> 下载: http://u.163.com/t0/bUAsgYsTI8W0
>  
> 预览: http://u.163.com/t0/GHwkOppxnZbj
>  
> 


Mime
View raw message