kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aki Yoshida <elak...@gmail.com>
Subject Re: When using MM2, how should the consumer be switched from source to target?
Date Thu, 17 Dec 2020 23:22:39 GMT
I'm answering to my own mail and would like to hear if this assumption
is correct.
it looks like I need 2.7.0 to have the automatic translation.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-545%3A+support+automated+consumer+offset+sync+across+clusters+in+MM+2.0
for earlier versions, there is no way to use the console client but
use java API RemoteClusterUtils.translateOffsets() to get the offset
and re-consuming from the new topic.
Is this correct?

El jue, 17 dic 2020 a las 21:18, Aki Yoshida (<elakito@gmail.com>) escribió:
>
> Hi,
> I have a question regarding how to migrate a consumer when the
> subscribing topic has been migrated to the target Kafka broker.
> Suppose a consumer is consuming from topic1 at the source Kafka
> broker. When MM2 mirrors this topic using options
> source.cluster.alias="", replication.policy.separator="", topic named
> topic1 shows up at the target Kafka broker.
>
> When the consumer instance simply switches the bootstrap server to
> start consuming from this topic1 at the target broker, the consumer
> seems to start subscribing from the latest offset. I thought the
> consumer offsets were also migrated to the target and the consumer
> could simply start subscribing from the mirrored topic from the
> continued offset. Did I miss some configuration parameters or does the
> consumer need to perform some actions to be able to consume the
> records seamlessly?
>
> I appreciate for your help. Thanks.
>
> regards, aki

Mime
View raw message