kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Curtin <curtin.ch...@gmail.com>
Subject Re: 0.8.0 HEAD 3/4/2013 performance jump?
Date Tue, 05 Mar 2013 13:30:01 GMT
Hi Jun,

I wasn't explicitly setting the ack anywhere.

Am I reading the code correctly that in SyncProducerConfig.scala the
DefaultRequiredAcks is 0? Thus not waiting on the leader?

Setting:  props.put("request.required.acks", "1"); causes the writes to go
back to the performance I was seeing before yesterday.

Are you guys open to changing the default to be 1? The MongoDB Java-driver
guys made a similar default change at the end of last year because many
people didn't understand the risk that the default value of no-ack was
putting them in until they had a node failure. So they default to 'safe'
and let you decide what your risk level is vs. assuming you can lose data.

Thanks,

Chris



On Tue, Mar 5, 2013 at 1:00 AM, Jun Rao <junrao@gmail.com> wrote:

> Chris,
>
> On the producer side, are you using ack=0? Earlier, ack=0 is the same as
> ack=1, which means that the producer has to wait for the message to be
> received by the leader. More recently, we did the actual implementation of
> ack=0, which means the producer doesn't wait for the message to reach the
> leader and therefore it is much faster.
>
> Thanks,
>
> Jun
>
> On Mon, Mar 4, 2013 at 12:01 PM, Chris Curtin <curtin.chris@gmail.com
> >wrote:
>
> > Hi,
> >
> > I'm definitely not complaining, but after upgrading to HEAD today my
> > producers are running much, much faster.
> >
> > Don't have any measurements, but last release I was able to tab windows
> to
> > stop a Broker before I could generate 500 partitioned messages. Now it
> > completes before I can get the Broker shutdown!
> >
> > Anything in particular you guys fixed?
> >
> > (I did remove all the files on disk per the email thread last week and
> > reset the ZooKeeper meta, but that shouldn't matter right?)
> >
> > Very impressive!
> >
> > Thanks,
> >
> > Chris
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message