kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elliot West <tea...@gmail.com>
Subject Re: Topic migration patterns
Date Thu, 11 Jul 2019 12:59:32 GMT
Thanks for your reply.

Yes, you can assume that nothing is shared between clusters.

There is no specific topic. I'm trying to establish general patterns that
can be applied to achieve this.

Thanks,

Elliot.

On Thu, 11 Jul 2019 at 10:50, Mich Talebzadeh <mich.talebzadeh@gmail.com>
wrote:

> Hi Elliot,
>
> As you are moving the topic from one cluster to another, I assume it
> implies a new zookeeper ensemble plus sets of new brokers?
>
> Can you describe the current topic?
>
> ${KAFKA_HOME}/bin/kafka-topics.sh --describe --zookeeper <HOST:PORT>,
> <HOST:PORT>,  --topic <TOPIC>
>
> HTH
>
> Dr Mich Talebzadeh
>
>
>
> LinkedIn *
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> <
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >*
>
>
>
> http://talebzadehmich.wordpress.com
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Thu, 11 Jul 2019 at 09:50, Elliot West <teabot@gmail.com> wrote:
>
> > Hello,
> >
> > I like to understand what strategies can be applied to migrate events,
> > producers, and consumers from one topic to another. Typically I'm
> thinking
> > of cases where:
> >
> >
> >    - we wish to migrate a topic from one cluster to another
> >    - high availability - minimise amount of time topic is unavailable to
> >    producers/consumers
> >    - no data loss (at least once)
> >    - avoid excessive latency (keep events flowing - no 'stopping the
> > world')
> >    - preserve event ordering wrt to partitions
> >    - there is a significant retention interval (days) that we wish to
> move
> >    to the destination topic also
> >
> >
> > I can imagine that tools such as MM1+2, Replicator, etc. are useful in
> > these circumstances. However, I'm more interested in orchestration
> > patterns: what things need to be moved, in what order, and subject to
> what
> > conditions?
> >
> > Thanks for your time,
> >
> > Elliot.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message