kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryanne Dolan <ryannedo...@gmail.com>
Subject Re: Kafka Mirror Maker 2: RemoteClusterUtils.translateOffsets() returning empty map
Date Tue, 04 Aug 2020 14:55:24 GMT
Sunny, is it possible there are no consumer groups? There will be no
checkpoints, and thus nothing to use for offset translation, if there are
no upstream consumer groups.

Ryanne

On Tue, Aug 4, 2020, 9:28 AM Sunny Lohani <sunny.lohani@gmail.com> wrote:

> Hi,
>
> I have 2 data centers, each having single node Zookeeper and Kafka cluster.
> I have a topic (single partition) in both the data center kafka clusters. I
> am using MM 2.0 as a dedicated cluster for bi-directional replication of
> the topic as well as using RemoteClusterUtils.translateOffsets() in my
> application for offset translation during failover. But the method is
> returning an empty map due to which the consumer is not resuming from
> proper offsets for local/remote topics.
>
> When I investigated further, I found that the checkpoint
> topics A.checkpoints.internal and B.checkpoints.internal in respective
> clusters do not have any kafka message. I don't see any errors in the
> mirror maker console logs. I searched everywhere on the internet but could
> not get any help. Below is the mm.properties:
>
> clusters = A, B
> A.bootstrap.servers = 10.34.45.113:19092
> B.bootstrap.servers = 10.34.45.113:29092
>
> A->B.enabled = true
> A->B.topics = .*
> B->A.enabled = true
> B->A.topics = .*
>
> # Setting replication factor of newly created remote topics
> replication.factor=1
>
> checkpoints.topic.replication.factor=1
> heartbeats.topic.replication.factor=1
> offset-syncs.topic.replication.factor=1
>
> offset.storage.replication.factor=1
> status.storage.replication.factor=1
> config.storage.replication.factor=1
>
> sync.topic.acls.enabled = false
>
> emit.checkpoints.enabled = true
> emit.checkpoints.interval.seconds = 5
> ----
>
> Need help on this urgently. Thanks in advance.
>
> Thanks & Regards,
> Sunny Kumar Lohani,
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message