kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan En <Ryan_...@hotmail.com>
Subject Mirrormaker producing to only one partition in a topic
Date Tue, 29 May 2018 20:28:40 GMT
Hi, I'm using Kafka version 0.10.2.0 and trying to use Mirrormaker to the messages from one
Kafka cluster to another.


The source and target Kafka cluster are pretty much set up the same... replication factor
is 3, number of partitions is 3, auto.create.topics.enable is true.


I am finding that in the target Kafka cluster, all the messages produced by Mirrormaker are
going into one partition only.


Here are my configs:


mirrormaker producer


security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="xxxxxxxx"
password="xxxxxxxxxx";
ssl.truststore.location=/path/to/truststore.jks
ssl.truststore.password=xxxxxxxxxxx
ssl.keystore.location=/path/to/keystore.jks
ssl.keystore.password=xxxxxxxxx
ssl.key.password=xxxxxxxx
retries=5000000
max.in.flight.requests.per.connection=1
client.id=xxxxxxxx
compression.type=lz4
bootstrap.servers=<aggregate-broker>:9292
acks=all


mirrormaker consumer


security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="xxxxxxxx"
password="xxxxxxxxxx";
ssl.truststore.location=/path/to/truststore.jks
ssl.truststore.password=xxxxxxxxxx
ssl.keystore.location=/path/to/keystore.jks
ssl.keystore.password=xxxxxxxxxx
ssl.key.password=xxxxxxxxx
client.id=xxxxxxxxxxxx
partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor
auto.commit.interval.ms=1000
auto.offset.reset=earliest
bootstrap.servers=<local-broker>:9192
session.timeout.ms=30000
group.id=xxxxxxxxxxxxxx
enable.auto.commit=false


Any ideas why this may happen?


Thanks in advance

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message