kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Skip Montanaro <skip.montan...@gmail.com>
Subject RE: Multiple brokers - do they share the load?
Date Fri, 01 Dec 2017 13:41:17 GMT
Are dynamically created topics thus never replicated?

On Nov 30, 2017 6:52 PM, "Tauzell, Dave" <Dave.Tauzell@surescripts.com>
wrote:

> You then also need to set this up for each topic you create:
>
> > bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 3 --partitions 3 --topic my-replicated-topic
>
> -Dave
>
> -----Original Message-----
> From: Skip Montanaro [mailto:skip.montanaro@gmail.com]
> Sent: Thursday, November 30, 2017 4:20 PM
> To: users@kafka.apache.org
> Subject: Re: Multiple brokers - do they share the load?
>
> > If you create a partitioned topic with at least 3 partitions then you
> will see your client connect to all of the brokers.  The client decides
> which partition a message should go to and then sends it directly to the
> broker that is the leader for that partition.  If you have replicated
> topics, then the brokers themselves will also be connected to one-another
> in order to replication messages.
>
> Thanks. So if I create three brokers all pointing to the same Zookeeper
> instance, then define
>
> offsets.topic.replication.factor=3
> transaction.state.log.replication.factor=3
>
> the brokers will find each other via Zookeeper and dynamically created
> topics (which is all I've used so far) will be properly replicated?
> Should I also bump transaction.state.log.min.isr above 1?
>
> Apologies for such basic questions. I'm still trying to absorb the
> documentation related to overall architecture and broker specifics.
>
> Skip
>
> This e-mail and any files transmitted with it are confidential, may
> contain sensitive information, and are intended solely for the use of the
> individual or entity to whom they are addressed. If you have received this
> e-mail in error, please notify the sender by reply e-mail immediately and
> destroy all copies of the e-mail and any attachments.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message