kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johnny Sheeley <shee...@aigee.org>
Subject Mirrormaker Re-partitioning Keyed Messages
Date Thu, 06 Aug 2015 00:23:55 GMT
Hi,
Based on the description of https://issues.apache.org/jira/browse/KAFKA-957
(along with a number of random sources in the documentation), I expected
keyed messages to remain in the same partition when using MirrorMaker. It
looks like there's been a lot of change since then.

I've been doing some testing, and it appears that MirrorMaker repartitions
these messages.

https://gist.github.com/sheeley/f2c718efb643add1ba3f

MirrorMaker is running with default settings, and the brokers are running
with almost-default, just changes to the zookeeper chroot, port, and
default partition count set to 6. This is with Kafka kafka_2.10-0.8.2.1.

Is this accurate? Am I missing something, or should I change my keys
somehow? Should I try testing with Java clients?
Thanks!
Johnny

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message