kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Visher <tim.vis...@gmail.com>
Subject Re: Any suggestion about Kafka/Zookeeper migration.
Date Thu, 04 Jan 2018 17:16:06 GMT
On Thu, Jan 4, 2018 at 11:55 AM, Tony Liu <jiangtao.liu@zuora.com> wrote:

> Thanks Timmy.
>
> I appreciate your suggestion, but for us, it's hard to do that , the
> problem are :
>
>    - 6+ Kafka cluster.
>    - more than 200 micro services or applications are talking with Kafka
>    now.
>
> Any other suggestion for our case ?
>

I've never done it but the standard migration guide should get you an
upgrade in place. It's just a terribly tedious sounding process.
https://kafka.apache.org/documentation/#upgrade

With that kind of topology I'd probably recommend getting something in
front of kafka that can do a cut over for you like HAProxy or DNS.
Obviously that's a future consideration though. For now, definitely go with
the in place migration.


>
>
> On Thu, Jan 4, 2018 at 6:06 AM, Tim Visher <tim.visher@gmail.com> wrote:
>
> > On Wed, Jan 3, 2018 at 11:39 PM, Tony Liu <jiangtao.liu@zuora.com>
> wrote:
> >
> > > This post here is aimed to ask experience about what did you do
> migration
> > > `Kafka/zookeeper` ?  :)
> > >
> > >
> > I think the easiest way to migrate kafka is to do a blue/green deploy.
> > Stand up an entire new cluster, cut over producers to it, wait for the
> old
> > cluster to drain, cut over consumers, then shut down the old cluster.
> >
> > Especially in AWS or any other virtualized environment this should be
> very
> > easy assuming that you don't need to deploy many things to do the
> cutover.
> > DNS or some other load balancing solution could help but you may be in
> too
> > deep to put that in place now.
> >
> > As far as migrating zookeeper, you could stand up a new zookeeper cluster
> > when spinning up your new kafka cluster and do the same thing.
> >
> > My team is currently doing exactly that right now.
> >
> > --
> >
> > In Christ,
> >
> > Timmy V.
> >
> > https://blog.twonegatives.com
> > https://five.sentenc.es
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message