kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Palino <tpal...@gmail.com>
Subject Re: Documentation typo for offsets.topic.replication.factor ?
Date Thu, 06 Aug 2015 01:39:13 GMT
That's exactly right. We've been talking about this internally at LinkedIn, and how to solve
it. I think the best option would be to have the broker throw an error on offset commits until
there are enough brokers to fulfill the configured RF.

We've seen this several times now when bootstrapping test and development clusters. You can
work around it by making sure all consumers are shut down until you have all the brokers started.
But that's not a good long term solution for this problem. 

-Todd


> On Aug 5, 2015, at 6:28 PM, James Cheng <jcheng@tivo.com> wrote:
> 
> Hi,
> 
> My kafka cluster has a __consumer_offsets topic with 50 partitions (the default for offsets.topic.num.partitions)
but with a replication factor of just 1 (the default for offsets.topic.replication.factor
should be 3).
> 
> From the docs http://kafka.apache.org/documentation.html:
> 
> offsets.topic.replication.factor    3    The replication factor for the offset commit
topic. A higher setting (e.g., three or four) is recommended in order to ensure higher availability.
If the offsets topic is created when fewer brokers than the replication factor then the offsets
topic will be created with fewer replicas.
> 
> 
> I'm guessing there's a typo there? I'm guessing it should be:
> 
> If the offsets topic is created when fewer brokers than the replication factor [are active],
then the offsets topic will be created with fewer replicas.
> 
> Or something along those lines?
> 
> Thanks,
> -James
> 
> 

Mime
View raw message