kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Snyder <tsny...@blackberry.com>
Subject RE: Any gotchas upgrading to 0.9?
Date Tue, 01 Dec 2015 18:24:01 GMT
The quota page is here: http://kafka.apache.org/documentation.html#design_quotas

"By default, each unique client-id receives a fixed quota in bytes/sec as configured by the
cluster (quota.producer.default, quota.consumer.default)"

I also noticed there's been a change in the replication configuration while reading: http://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/

It may not break anything, but it may impact how you decide to configure and monitor replication

"Now there is only one value you need to configure on the server which is replica.lag.time.max.ms.
The interpretation of this has changed to be the time for which a replica has been out-of-sync
with the leader. Stuck or failed replicas are detected the same way as before - if a replica
fails to send a fetch request for longer than replica.lag.time.max.ms, it is considered dead
and is removed from the ISR. The mechanism of detecting slow replicas has changed - if a replica
starts lagging behind the leader for longer than replica.lag.time.max.ms, then it is considered
too slow and is removed from the ISR. So even if there is a spike in traffic and large batches
of messages are written on the leader, unless the replica consistently remains behind the
leader for replica.lag.time.max.ms, it will not shuffle in and out of the ISR."

-----Original Message-----
From: Rajiv Kurian [mailto:rajiv@signalfx.com] 
Sent: Tuesday, December 01, 2015 11:57
To: users@kafka.apache.org
Subject: Re: Any gotchas upgrading to 0.9?

Also I remember reading (can't find now) something about default traffic
quotas. I'd hope the default quotas are very large (infinite?) and not
small so that compatibility is maintained. It would be very unfortunate if
some of our traffic was throttled because of the upgrade because of magic
defaults. For example we have a certain cluster dedicated to serving a
single important topic and we'd hate for it to be throttled because of
incorrect defaults.


On Tue, Dec 1, 2015 at 8:54 AM, Rajiv Kurian <rajiv@signalfx.com> wrote:

> I saw the upgrade path documentation at
> http://kafka.apache.org/documentation.html and that kind of answers (1).
> Not sure if there is anything about client compatibility though.
> On Tue, Dec 1, 2015 at 8:51 AM, Rajiv Kurian <rajiv@signalfx.com> wrote:
>> I plan to upgrade both the server and clients to 0.9. Had a few questions
>> before I went ahead with the upgrade:
>> 1. Do all brokers need to be on 0.9? Currently we are running 0.8.2. We'd
>> ideally like to convert only a few brokers to 0.9 and only if we don't see
>> problems convert the rest.
>> 2. Is it possible to run Kafka 0.9 clients (specifically the consumer)
>> with Kafka 0.8.2 brokers?
>> Any link to the upgrade path would be really useful.
>> Thanks,
>> Rajiv
View raw message