kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harper Henn <harper.h...@datto.com.INVALID>
Subject Re: A new way migrate replica from old broker to new one
Date Sun, 25 Aug 2019 18:36:51 GMT
Hi,

I've used the same method to upgrade the OS a Kafka cluster uses (also in
AWS). The steps were roughly:

- Create new EC2 instance, copy config files from the broker I'm going to
replace (broker ID stays the same, other fields might change if your IP
changes)
- Stop the broker I'm going to replace, detach volume with Kafka data logs
and re-attach to the new EC2 instance
- Start new broker on new EC2 instance, and it begins replicating from the
other brokers until it can become partition leader again

This was on a production cluster with a message-in rate of about ~1000
messages per broker per second.

Harper

On Wed, Aug 21, 2019 at 5:39 AM Xiangyuan LI <flashmouse2012@gmail.com>
wrote:

> thx!
>   we'll try this method soon.
>
> David Jacot <djacot@confluent.io> 于2019年8月21日周三 下午3:38写道:
>
> > Hello,
> >
> > Yes, that should be fine. If you move the data to a new machine and use
> the
> > corresponding broker.id, it is basically the same broker but on
> different
> > vm.
> >
> > Best,
> > David
> >
> > On Fri, Aug 16, 2019 at 10:03 AM Xiangyuan LI <flashmouse2012@gmail.com>
> > wrote:
> >
> > > Hi,
> > > Now our kafka cluster has 5 brokers that broker.id is 1,2,3,4,5.
> > >
> > > We want to decommission broker 1 and add new broker 6,we expect that
> all
> > > replica in broker 1 could migrate to broker 6,and nothing change for
> 2~5.
> > >
> > > Now we have an idea.our servers are in aws,we plan to remove  disk from
> > > broker 1,and add it to broker 6,then modify broker6's broker.id to
> 1,and
> > > then add it to cluster.
> > >
> > > now we have broker 2~6,but broker 6's broker.id is 1,i want to know
> > whther
> > > we could do so?
> > >
> > > thx..
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message