kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liam Clarke <liam.cla...@adscale.co.nz>
Subject Re: Batch vs buffer differences
Date Tue, 11 Sep 2018 07:29:14 GMT
Batch size is a threshold that determines when a batch sends.

Buffer size is total memory available to the producer. It determines how
many records the producer can store in memory to send.

While they relate somewhat in that buffer size couldn't be less than (batch
size * num partitions), you can set it higher to allow more buffering in
the producer.

Does that make sense? Batch size determines when a message composed of
batched records is sent. Buffer size controls producer memory.

On Tue, 11 Sep. 2018, 7:16 pm darekAsz, <darekkafka@gmail.com> wrote:

> Hi
> I still learn kafka, but in some cases documentation is not so clear to
> understand all things. Can someone more experienced explain me what is
> buffer and what is batch on producer side. I readed that producer stores
> messages in buffer before sending and also that batch mechanism is used for
> sending multiple messages in one request. So where are differences? For me
> this sounds very similar.
> Best regards

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message