kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tao xiao <xiaotao...@gmail.com>
Subject Re: New consumer not fetching as quickly as possible
Date Tue, 01 Dec 2015 10:35:21 GMT

In your case I think you can set fetch.min.bytes=1 so that the server will
answer the fetch request as soon as a single byte of data is available
instead of accumulating enough messages.

But in my case is I have plenty of messages in broker and I am sure the
size of total message are much larger than the default setting which is
1024 bytes but still the consumer doesn't return messages for every poll.

On Tue, 1 Dec 2015 at 18:29 Gerard Klijs <gerard.klijs@dizzit.com> wrote:

> I was experimenting with the timeout setting, but as long as messages are
> produced and the consumer(s) keep polling I saw little difference. I did
> see for example that when producing only 1 message a second, still it
> sometimes wait to get three messages. So I also would like to know if there
> is a faster way.
> On Tue, Dec 1, 2015 at 10:35 AM tao xiao <xiaotao183@gmail.com> wrote:
> > Hi team,
> >
> > I am using the new consumer with broker version 0.9.0. I notice that
> > poll(time) occasionally returns 0 message even though I have enough
> > messages in broker. The rate of returning 0 message is quite high like 4
> > out of 5 polls return 0 message. It doesn't help by increasing the poll
> > timeout from 300ms to 1 second. are there any configurations that I can
> > tune to fetch  data as quickly as possible?
> >
> > Both consumer and broker configs are default
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message