kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryanne Dolan <ryannedo...@gmail.com>
Subject Re: MirrorMaker 2.0 ( working off origin/2.4 )
Date Mon, 04 Nov 2019 20:08:03 GMT
> BTW any ideas when 2.4 is being released

Looks like there are a few blockers still.

On Mon, Nov 4, 2019 at 2:06 PM Vishal Santoshi <vishal.santoshi@gmail.com>
wrote:

> I bet I have tested the "b.producer.acks' route. I will test again and let
> you know. Note that I resorted to hardcoding that value in the Sender and
> that alleviated the throttle I was seeing on consumption.  BTW any ideas
> when 2.4 is being released ( I thought it was Oct 30th 2019 )...
>
> On Mon, Nov 4, 2019 at 1:49 PM Ryanne Dolan <ryannedolan@gmail.com> wrote:
>
> > Vishal, b.producer.acks should work, as can be seen in the following unit
> > test with similar producer property "client.id":
> >
> >
> >
> https://github.com/apache/kafka/blob/6b905ade0cdc7a5f6f746727ecfe4e7a7463a200/connect/mirror/src/test/java/org/apache/kafka/connect/mirror/MirrorMakerConfigTest.java#L182
> >
> > Keep in mind this should be configured for the target cluster, and it
> might
> > not take effect until the workers finish rebalancing.
> >
> > Ryanne
> >
> > On Sun, Nov 3, 2019, 9:40 AM Vishal Santoshi <vishal.santoshi@gmail.com>
> > wrote:
> >
> > > Hello folks,
> > >                   Was doing stress tests and realized that the
> > replication
> > > to the target cluster and thus the configuration of the KafkaProducer
> > has a
> > > default acks of -1 ( all ) and that was prohibitively expensive. It
> > should
> > > have been a simple a->b.producer.acks = 1 ( or  b.producer.acks = 1 or
> > acks
> > > = 1, have tried all ) override in the mm2.properties, but it does not
> > take.
> > > I verified by turning on TRACE and a little  instrumenting the code to
> > > print out the acks .
> > >
> > >
> >
> https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java#L806
> > > .
> > >  Am I missing something?
> > >
> > > Regards
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message