kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JIEFU GONG <jg...@berkeley.edu>
Subject Re: Changing Kafka configs for already running brokers
Date Wed, 05 Aug 2015 19:48:05 GMT
Got it, thanks for your help Grant!

On Wed, Aug 5, 2015 at 10:49 AM, Grant Henke <ghenke@cloudera.com> wrote:

> For any of the configs I can think of changing; this approach of taking no
> outage should work. However, I would be sure to keep all of your configs
> uniform instead of just changing the leader. Therefore the process would
> look something like:
>
> 1) Alter the config files for all brokers (using the config management tool
> of choice)
> 2) Perform a rolling restart of the cluster 1 broker at a time
>        - Make sure controlled.shutdown.enable=true (docs here
> <http://kafka.apache.org/documentation.html#basic_ops_restarting>)
>        - Make sure you move any partitions with replication=1 so that you
> don't take an outage (or better yet have a higher replication in general)
> 3) Rebalance leaders (docs here
> <http://kafka.apache.org/documentation.html#basic_ops_leader_balancing>)
>        - Or enable auto.leader.rebalance.enable=true and it will
> automatically happen for you
>
> Thanks,
> Grant
>
> On Wed, Aug 5, 2015 at 12:26 PM, JIEFU GONG <jgong@berkeley.edu> wrote:
>
> > Hi all,
> >
> > According to https://issues.apache.org/jira/browse/KAFKA-1229, there is
> > currently no support and no reason to support dynamic configuration
> changes
> > for Kafka servers that are already running. I just wanted to solidify my
> > understanding of how configs would then be changed, specifically:
> >
> > If i wanted to change configs on a running cluster (say log retention)
> but
> > I do not want unavailability for that cluster at any time, would the
> right
> > way of changing the configs be:
> >
> > 1) Alter the config file
> > 2) Bring down the leader broker using a controlled shutdown, letting
> > replicas handle the load while the leader is down
> > 3) Restart the leader broker with new configs, using preferred leader
> > assignment to have it resume control
> >
> > Would I also need to bring down the in sync replicas and reboot them too?
> > Thanks for the help!
> >
> > --
> >
> > Jiefu Gong
> > University of California, Berkeley | Class of 2017
> > B.A Computer Science | College of Letters and Sciences
> >
> > jgong@berkeley.edu <eliselim@berkeley.edu> | (925) 400-3427
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>



-- 

Jiefu Gong
University of California, Berkeley | Class of 2017
B.A Computer Science | College of Letters and Sciences

jgong@berkeley.edu <eliselim@berkeley.edu> | (925) 400-3427

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message