kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Coldham <dani...@wix.com>
Subject Increase partitions and replication of __consumer_offsets
Date Tue, 23 Jun 2015 12:15:24 GMT
Hi all,
I'm using Kafka 0.8.2.1 in production.

My Kafka Config is pretty much vanilla, so (as far as I understand) offsets
are being written to Zookeeper.
As recommended, I want to start writing offsets to Kafka instead of
Zookeeper.

I was surprised to see that the __consumer_offsets topic already exists.
But let's assume that's ok.
The topic has 50 partitions (which is the default) and a replication factor
of 1. Of course, I only had 1 broker to begin with so it makes sense that
the replication factor is 1 and not 3 (which is the default).
Now that I've added more brokers, I want to increase the number of
partitions to 200 (as recommended) and the replication factor to 3 (also as
recommended).

I'd like to do this before starting the process of migrating offsets to
Kafka.

What is the best way of increasing the partitions and replication factor of
__consumer_offsets?

Thanks,
*Daniel Coldham*


N.B.
Recommendations taken from:
http://kafka.apache.org/documentation.html
https://cwiki.apache.org/confluence/display/KAFKA/FAQ
http://www.slideshare.net/jjkoshy/offset-management-in-kafka

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message