kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aseem Bansal <asmbans...@gmail.com>
Subject Re: Topic discovery when supporting multiple kafka clusters
Date Tue, 06 Dec 2016 12:04:56 GMT
What configurations allow you to assign topics to specific brokers?

I can see https://kafka.apache.org/documentation#basic_ops_automigrate.
This should allow you to move things around but does that keep anything
from being re-assigned to the old ones?

On Tue, Dec 6, 2016 at 5:25 PM, Asaf Mesika <asaf.mesika@gmail.com> wrote:

> Why not re-use same cluster? You can assign topics to be live only within a
> specific set of brokers. Thus you have one "bus" for messages, simplifying
> your applications code and configurations
>
> On Mon, Dec 5, 2016 at 9:43 PM Yifan Ying <nafan628@gmail.com> wrote:
>
> > Hi,
> >
> > Initially, we have only one Kafka cluster shared across all teams. But
> now
> > this cluster is very close to out of resources (disk space, # of
> > partitions, etc.). So we are considering adding another Kafka cluster.
> But
> > what's the best practice of topic discovery, so that applications know
> > which cluster their topics live? We have been using Zookeeper for service
> > discovery, maybe it's also good for this purpose?
> >
> > Thanks
> >
> > --
> > Yifan
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message