kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Henke <ghe...@cloudera.com>
Subject Re: Changing Kafka configs for already running brokers
Date Wed, 05 Aug 2015 17:49:08 GMT
For any of the configs I can think of changing; this approach of taking no
outage should work. However, I would be sure to keep all of your configs
uniform instead of just changing the leader. Therefore the process would
look something like:

1) Alter the config files for all brokers (using the config management tool
of choice)
2) Perform a rolling restart of the cluster 1 broker at a time
       - Make sure controlled.shutdown.enable=true (docs here
<http://kafka.apache.org/documentation.html#basic_ops_restarting>)
       - Make sure you move any partitions with replication=1 so that you
don't take an outage (or better yet have a higher replication in general)
3) Rebalance leaders (docs here
<http://kafka.apache.org/documentation.html#basic_ops_leader_balancing>)
       - Or enable auto.leader.rebalance.enable=true and it will
automatically happen for you

Thanks,
Grant

On Wed, Aug 5, 2015 at 12:26 PM, JIEFU GONG <jgong@berkeley.edu> wrote:

> Hi all,
>
> According to https://issues.apache.org/jira/browse/KAFKA-1229, there is
> currently no support and no reason to support dynamic configuration changes
> for Kafka servers that are already running. I just wanted to solidify my
> understanding of how configs would then be changed, specifically:
>
> If i wanted to change configs on a running cluster (say log retention) but
> I do not want unavailability for that cluster at any time, would the right
> way of changing the configs be:
>
> 1) Alter the config file
> 2) Bring down the leader broker using a controlled shutdown, letting
> replicas handle the load while the leader is down
> 3) Restart the leader broker with new configs, using preferred leader
> assignment to have it resume control
>
> Would I also need to bring down the in sync replicas and reboot them too?
> Thanks for the help!
>
> --
>
> Jiefu Gong
> University of California, Berkeley | Class of 2017
> B.A Computer Science | College of Letters and Sciences
>
> jgong@berkeley.edu <eliselim@berkeley.edu> | (925) 400-3427
>



-- 
Grant Henke
Software Engineer | Cloudera
grant@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message