kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: Any gotchas upgrading to 0.9?
Date Tue, 01 Dec 2015 19:39:53 GMT
Old clients should work well with the new servers, while vice versa is not
generally true mainly because the new consumer client uses a few new
request types that only the new brokers can recognize. So the normal
upgrade path is server-first, clients later.

Filed https://issues.apache.org/jira/browse/KAFKA-2923 to update the
upgrade doc to include it.

Guozhang

On Tue, Dec 1, 2015 at 11:14 AM, Rajiv Kurian <rajiv@signalfx.com> wrote:

> Thanks folks. Anywhere I can read about client compatibility i.e. old
> clients working with new servers and new clients working with old servers?
>
> Thanks,
> Rajiv
>
> On Tue, Dec 1, 2015 at 10:54 AM, Jay Kreps <jay@confluent.io> wrote:
>
> > I think the point is that we should ideally try to cover all these in the
> > "upgrade" notes.
> >
> > -Jay
> >
> > On Tue, Dec 1, 2015 at 10:37 AM, Aditya Auradkar <aauradkar@linkedin.com
> >
> > wrote:
> >
> > > Rajiv,
> > >
> > > By default, the quota is unlimited until you decide to configure them
> > > explicitly.
> > > And yes, we did get rid of "replica.lag.max.messages", so that
> > > configuration will no longer apply.
> > >
> > > Aditya
> > >
> > > On Tue, Dec 1, 2015 at 10:24 AM, Todd Snyder <tsnyder@blackberry.com>
> > > wrote:
> > >
> > > > The quota page is here:
> > > > http://kafka.apache.org/documentation.html#design_quotas
> > > >
> > > > "By default, each unique client-id receives a fixed quota in
> bytes/sec
> > as
> > > > configured by the cluster (quota.producer.default,
> > > quota.consumer.default)"
> > > >
> > > >
> > > > I also noticed there's been a change in the replication configuration
> > > > while reading:
> > > >
> > >
> >
> http://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/
> > > >
> > > > It may not break anything, but it may impact how you decide to
> > configure
> > > > and monitor replication
> > > >
> > > > "Now there is only one value you need to configure on the server
> which
> > is
> > > > replica.lag.time.max.ms. The interpretation of this has changed to
> be
> > > the
> > > > time for which a replica has been out-of-sync with the leader. Stuck
> or
> > > > failed replicas are detected the same way as before - if a replica
> > fails
> > > to
> > > > send a fetch request for longer than replica.lag.time.max.ms, it is
> > > > considered dead and is removed from the ISR. The mechanism of
> detecting
> > > > slow replicas has changed - if a replica starts lagging behind the
> > leader
> > > > for longer than replica.lag.time.max.ms, then it is considered too
> > slow
> > > > and is removed from the ISR. So even if there is a spike in traffic
> and
> > > > large batches of messages are written on the leader, unless the
> replica
> > > > consistently remains behind the leader for replica.lag.time.max.ms,
> it
> > > > will not shuffle in and out of the ISR."
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Rajiv Kurian [mailto:rajiv@signalfx.com]
> > > > Sent: Tuesday, December 01, 2015 11:57
> > > > To: users@kafka.apache.org
> > > > Subject: Re: Any gotchas upgrading to 0.9?
> > > >
> > > > Also I remember reading (can't find now) something about default
> > traffic
> > > > quotas. I'd hope the default quotas are very large (infinite?) and
> not
> > > > small so that compatibility is maintained. It would be very
> unfortunate
> > > if
> > > > some of our traffic was throttled because of the upgrade because of
> > magic
> > > > defaults. For example we have a certain cluster dedicated to serving
> a
> > > > single important topic and we'd hate for it to be throttled because
> of
> > > > incorrect defaults.
> > > >
> > > > Thanks,
> > > > Rajiv
> > > >
> > > > On Tue, Dec 1, 2015 at 8:54 AM, Rajiv Kurian <rajiv@signalfx.com>
> > wrote:
> > > >
> > > > > I saw the upgrade path documentation at
> > > > > http://kafka.apache.org/documentation.html and that kind of
> answers
> > > (1).
> > > > > Not sure if there is anything about client compatibility though.
> > > > >
> > > > >
> > > > > On Tue, Dec 1, 2015 at 8:51 AM, Rajiv Kurian <rajiv@signalfx.com>
> > > wrote:
> > > > >
> > > > >> I plan to upgrade both the server and clients to 0.9. Had a few
> > > > questions
> > > > >> before I went ahead with the upgrade:
> > > > >>
> > > > >> 1. Do all brokers need to be on 0.9? Currently we are running
> 0.8.2.
> > > > We'd
> > > > >> ideally like to convert only a few brokers to 0.9 and only if
we
> > don't
> > > > see
> > > > >> problems convert the rest.
> > > > >>
> > > > >> 2. Is it possible to run Kafka 0.9 clients (specifically the
> > consumer)
> > > > >> with Kafka 0.8.2 brokers?
> > > > >>
> > > > >> Any link to the upgrade path would be really useful.
> > > > >>
> > > > >> Thanks,
> > > > >> Rajiv
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message