kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Melville <amelvi...@g.hmc.edu>
Subject Can Mirroring Preserve Every Topic's Partition?
Date Wed, 26 Nov 2014 04:19:54 GMT
Howdy friends,


I'd like to mirror the topics on several clusters to a central cluster, and
I'm looking at using the default Mirrormaker to do so. I've already done
some basic testing on the Mirrormaker found here:

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27846330

and managed to successfully copy a topic's partitions on a source cluster
to a topic on a target cluster. So I'm able to mirror correctly. However
for my particular use case I need to ensure that when I copy a topic's
partitions from source cluster to target cluster, a partition created on
the target cluster contains data in the exact same order as the data on the
corresponding partition on the source cluster.

I'm thinking of writing a Simple Consumer so I can manually compare the
events in a source cluster's partition with the corresponding partition on
the target cluster, but I'm not 100% sure if I'll be able to verify my
guarantee if I do it this way. Can anyone here verify that partitions
copied over to the target cluster by the default Mirrormaker are an exact
copy of those on the source cluster?


Thanks in advance,

Alex Melville

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message