kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maxime Nay <maxime...@gmail.com>
Subject Re: Add a new node in an existing kafka 0.8 cluster
Date Mon, 02 Dec 2013 22:12:03 GMT
Ok, thanks.
And now, let's say that I want to add two new nodes to increase the
capacity of our cluster. Is that possible with the current beta1 release ?


2013/12/2 Neha Narkhede <neha.narkhede@gmail.com>

> You can keep the broker.id of the new node same as the old node. Then it
> will start up and copy everything from the leader for the partitions it is
> assigned to. After it is caught up, you can run the
>
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-2.PreferredReplicaLeaderElectionTooltool
> to balance the leaders, so that the new node will have a few leaders
> and will start serving data.
>
> Thanks,
> Neha
>
>
> On Mon, Dec 2, 2013 at 1:50 PM, Maxime Nay <maximenay@gmail.com> wrote:
>
> > Hi,
> >
> > Last week we set-up a new kafka 0.8 cluster, using the beta1 released
> > available here : http://kafka.apache.org/downloads.html
> >
> > It worked fine until we tried to replace a node in this cluster.
> > We shutdown a node, then bring up a new one. The new node is registered
> in
> > zookeeper, but it doesn't get any data. (We have 10 nodes, 10 partitions
> > for each topic, and a replication factor of 3.)
> >
> > We saw that there are some tools available to balance the load, and add
> > some broker to an existing cluster (
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-ClusterExpansion
> > ).
> > But it seems they are not yet included in the Kafka version that we
> > download from kafka.apache.org.
> >
> > Is there any branch where we can get a more current - production safe
> > version where we can find the latest tools? Is it safe to get code from
> > trunk and deploy it in production?
> >
> > Thanks
> > Maxime
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message