kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vishal Santoshi <vishal.santo...@gmail.com>
Subject Re: MirrorMaker 2.0 ( working off origin/2.4 )
Date Mon, 04 Nov 2019 20:06:14 GMT
I bet I have tested the "b.producer.acks' route. I will test again and let
you know. Note that I resorted to hardcoding that value in the Sender and
that alleviated the throttle I was seeing on consumption.  BTW any ideas
when 2.4 is being released ( I thought it was Oct 30th 2019 )...

On Mon, Nov 4, 2019 at 1:49 PM Ryanne Dolan <ryannedolan@gmail.com> wrote:

> Vishal, b.producer.acks should work, as can be seen in the following unit
> test with similar producer property "client.id":
>
>
> https://github.com/apache/kafka/blob/6b905ade0cdc7a5f6f746727ecfe4e7a7463a200/connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorMakerConfigTest.java#L182
>
> Keep in mind this should be configured for the target cluster, and it might
> not take effect until the workers finish rebalancing.
>
> Ryanne
>
> On Sun, Nov 3, 2019, 9:40 AM Vishal Santoshi <vishal.santoshi@gmail.com>
> wrote:
>
> > Hello folks,
> >                   Was doing stress tests and realized that the
> replication
> > to the target cluster and thus the configuration of the KafkaProducer
> has a
> > default acks of -1 ( all ) and that was prohibitively expensive. It
> should
> > have been a simple a->b.producer.acks = 1 ( or  b.producer.acks = 1 or
> acks
> > = 1, have tried all ) override in the mm2.properties, but it does not
> take.
> > I verified by turning on TRACE and a little  instrumenting the code to
> > print out the acks .
> >
> >
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L806
> > .
> >  Am I missing something?
> >
> > Regards
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message