kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryanne Dolan <ryannedo...@gmail.com>
Subject Re: MirrorMaker2 to mirror new topics?
Date Mon, 09 Mar 2020 16:51:59 GMT
Hey Peter, often the problem is the replication factor or min ISRs being
misconfigured, which can prevent MM2 (or anything else) from sending. Hope
that helps!

Ryanne

On Mon, Mar 9, 2020, 10:35 AM Péter Sinóros-Szabó
<peter.sinoros-szabo@transferwise.com.invalid> wrote:

> Hi,
>
> it is mirroring but the problem was that on of the task failed with not
> being able to produce messages to the destination cluster and the task
> stopped there.
>
> Peter
>
> On Mon, 9 Mar 2020 at 11:20, Péter Sinóros-Szabó <
> peter.sinoros-szabo@transferwise.com> wrote:
>
> > Hi,
> >
> > Should MM2 automatically start to mirror new topics?
> > I see in the logs that MM2 created the remote topic, but it is not
> > mirroring the new messages.
> >
> > In the source cluster, topic was created 3 days ago and there is about
> 200
> > million messages in it.
> > Just to test if the messages are there, I could consume 30.000 messages
> > from it (just didn't want to read more...).
> >
> > In the destination cluster, I see the topic is there, MM2 logs that it
> > created it, MM2 logs also that it is consuming from all 6 partitions of
> it
> > besides the other about 600 topics that is is mirroring.
> >
> > But on the destination cluster, I do not see new messages arriving and
> > also in total I only see about 22.000 messages.
> >
> > What can be the problem here?
> >
> > Thanks,
> > Peter
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message