kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priya Matpadi <priya.matp...@ecofactor.com>
Subject Re: Purgatory
Date Fri, 01 Nov 2013 06:45:22 GMT
Guozhang,
The documentation is not very clear.
Marc's response for producer purgatory makes sense.
I am not entirely clear on fetch purgatory.
How does broker use purgatory? Is it a temporary holding area? What happens
to the messages if purge interval is exceeded in case of either/both
producer and consumer? Are messages dropped in this case?
Thanks,
Priya


On Thu, Oct 31, 2013 at 2:47 PM, Guozhang Wang <wangguoz@gmail.com> wrote:

> Hello Priya,
>
> You can find the definitions of these two configs here:
>
> http://kafka.apache.org/documentation.html#brokerconfigs
>
> Guozhang
>
>
> On Thu, Oct 31, 2013 at 11:20 AM, Marc Labbe <mrlabbe@gmail.com> wrote:
>
> > Hi Priya
> >
> > my understanding is producer requests will be delayed (and put in request
> > purgatory) only if your producer uses ack=-1. It will be in the purgatory
> > (delayed) until all brokers have acknowledged the messages to be
> > replicated. The documentation suggests to monitor the
> > ProducerRequestPurgatory size metrics , but it only applies if you're
> using
> > ack=-1, otherwise, this value will always be 0.
> >
> > For consumer requests, they'll be in purgatory (delayed) until the max
> > allowed time to respond has been reached, unless it has enough messages
> to
> > fill the buffer before that. The request will not end up in the purgatory
> > if you're making a blocking request (max wait <= 0).
> >
> > Not sure about the configuration interval though.
> >
> > marc
> >
> >
> > On Thu, Oct 31, 2013 at 12:41 PM, Priya Matpadi <
> > priya.matpadi@ecofactor.com
> > > wrote:
> >
> > > Hello,
> > > What is purgatory? I believe the following two properties relate to
> > > consumer and producer respectively.
> > > Could someone please explain the significance of these?
> > > fetch.purgatory.purge.interval.requests=100
> > > producer.purgatory.purge.interval.requests=100
> > >
> > > Thanks,
> > > Priya
> > >
> >
>
>
>
> --
> -- Guozhang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message