From users-return-7510-apmail-kafka-users-archive=kafka.apache.org@kafka.apache.org Mon Dec 2 21:53:55 2013 Return-Path: X-Original-To: apmail-kafka-users-archive@www.apache.org Delivered-To: apmail-kafka-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5855F10692 for ; Mon, 2 Dec 2013 21:53:55 +0000 (UTC) Received: (qmail 86363 invoked by uid 500); 2 Dec 2013 21:53:54 -0000 Delivered-To: apmail-kafka-users-archive@kafka.apache.org Received: (qmail 86341 invoked by uid 500); 2 Dec 2013 21:53:54 -0000 Mailing-List: contact users-help@kafka.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@kafka.apache.org Delivered-To: mailing list users@kafka.apache.org Received: (qmail 86332 invoked by uid 99); 2 Dec 2013 21:53:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Dec 2013 21:53:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of neha.narkhede@gmail.com designates 74.125.82.182 as permitted sender) Received: from [74.125.82.182] (HELO mail-we0-f182.google.com) (74.125.82.182) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Dec 2013 21:53:48 +0000 Received: by mail-we0-f182.google.com with SMTP id q59so12541962wes.41 for ; Mon, 02 Dec 2013 13:53:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=t6xyYh1tDijZ2kq846qnjeSt+uuT2spNnwLLLvR23SY=; b=x4PdhlZNgiJxsWQLmkf9f0vuyUDTDsIfDJ0q1pHjB60AqVtmL/Ut3q5JJwgOLJoR5H DIqM20uM7i/atG48fYJjd45uh/h1ZnbEC0ENV2sKVqwDz+D4sYGgAWsg/keWXU1rFbzx itfqxMdWXA+sjvGGNYkoBCXhmgk0sLK4ld36QNikNThPNh6GPIaUWb2jkwIppJQkty+7 NPbb+tKuNqupbPeUlFJBBN1TbyAU/TDH+9QRL0qgz2mRyRRpP7IMbY38fz2/7CZ7th0J xL2jPuFIxrBBM65PqPhT1sXsadIRTb1610CurdRCCTJTYwi8ZljfQDy3IQiU0b5OJ1Pf ysxQ== MIME-Version: 1.0 X-Received: by 10.194.93.193 with SMTP id cw1mr24570106wjb.38.1386021207906; Mon, 02 Dec 2013 13:53:27 -0800 (PST) Received: by 10.216.53.7 with HTTP; Mon, 2 Dec 2013 13:53:27 -0800 (PST) In-Reply-To: References: Date: Mon, 2 Dec 2013 13:53:27 -0800 Message-ID: Subject: Re: Add a new node in an existing kafka 0.8 cluster From: Neha Narkhede To: "users@kafka.apache.org" Content-Type: multipart/alternative; boundary=047d7bdc0538bf9e7804ec943582 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bdc0538bf9e7804ec943582 Content-Type: text/plain; charset=ISO-8859-1 You can keep the broker.id of the new node same as the old node. Then it will start up and copy everything from the leader for the partitions it is assigned to. After it is caught up, you can run the https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-2.PreferredReplicaLeaderElectionTooltool to balance the leaders, so that the new node will have a few leaders and will start serving data. Thanks, Neha On Mon, Dec 2, 2013 at 1:50 PM, Maxime Nay wrote: > Hi, > > Last week we set-up a new kafka 0.8 cluster, using the beta1 released > available here : http://kafka.apache.org/downloads.html > > It worked fine until we tried to replace a node in this cluster. > We shutdown a node, then bring up a new one. The new node is registered in > zookeeper, but it doesn't get any data. (We have 10 nodes, 10 partitions > for each topic, and a replication factor of 3.) > > We saw that there are some tools available to balance the load, and add > some broker to an existing cluster ( > > https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-ClusterExpansion > ). > But it seems they are not yet included in the Kafka version that we > download from kafka.apache.org. > > Is there any branch where we can get a more current - production safe > version where we can find the latest tools? Is it safe to get code from > trunk and deploy it in production? > > Thanks > Maxime > --047d7bdc0538bf9e7804ec943582--