kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liam Clarke <liam.cla...@adscale.co.nz>
Subject Re: Batch vs buffer differences
Date Tue, 11 Sep 2018 07:45:16 GMT
Just to clarify - a batch of records is sent when batch.size or linger.ms
are hit - so to control messaging rates/sizes you tweak both of those.

On Tue, 11 Sep. 2018, 7:38 pm darekAsz, <darekkafka@gmail.com> wrote:

> Thank you so much :)
> I think that now all is clear for me
>
> wt., 11 wrz 2018 o 09:28 Liam Clarke <liam.clarke@adscale.co.nz>
> napisaƂ(a):
>
> > Batch size is a threshold that determines when a batch sends.
> >
> > Buffer size is total memory available to the producer. It determines how
> > many records the producer can store in memory to send.
> >
> > While they relate somewhat in that buffer size couldn't be less than
> (batch
> > size * num partitions), you can set it higher to allow more buffering in
> > the producer.
> >
> > Does that make sense? Batch size determines when a message composed of
> > batched records is sent. Buffer size controls producer memory.
> >
> >
> > On Tue, 11 Sep. 2018, 7:16 pm darekAsz, <darekkafka@gmail.com> wrote:
> >
> > > Hi
> > > I still learn kafka, but in some cases documentation is not so clear to
> > > understand all things. Can someone more experienced explain me what is
> > > buffer and what is batch on producer side. I readed that producer
> stores
> > > messages in buffer before sending and also that batch mechanism is used
> > for
> > > sending multiple messages in one request. So where are differences? For
> > me
> > > this sounds very similar.
> > > Best regards
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message