kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Coldham <dani...@wix.com>
Subject Re: Increase partitions and replication of __consumer_offsets
Date Tue, 23 Jun 2015 13:38:45 GMT
To make my question clearer:
I know how to increase the partitions and the replication factor of any
plain old topic.
I'm worried that making changes to this "internal" topic could cause
problems, so I'm looking for advice.

*Daniel Coldham*

On Tue, Jun 23, 2015 at 3:15 PM, Daniel Coldham <danielc@wix.com> wrote:

> Hi all,
> I'm using Kafka in production.
> My Kafka Config is pretty much vanilla, so (as far as I understand)
> offsets are being written to Zookeeper.
> As recommended, I want to start writing offsets to Kafka instead of
> Zookeeper.
> I was surprised to see that the __consumer_offsets topic already exists.
> But let's assume that's ok.
> The topic has 50 partitions (which is the default) and a replication
> factor of 1. Of course, I only had 1 broker to begin with so it makes sense
> that the replication factor is 1 and not 3 (which is the default).
> Now that I've added more brokers, I want to increase the number of
> partitions to 200 (as recommended) and the replication factor to 3 (also as
> recommended).
> I'd like to do this before starting the process of migrating offsets to
> Kafka.
> What is the best way of increasing the partitions and replication factor
> of __consumer_offsets?
> Thanks,
> *Daniel Coldham*
> N.B.
> Recommendations taken from:
> http://kafka.apache.org/documentation.html
> https://cwiki.apache.org/confluence/display/KAFKA/FAQ
> http://www.slideshare.net/jjkoshy/offset-management-in-kafka

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message