kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tejas Patil <tejas.patil...@gmail.com>
Subject Re: Random Broker/Partition for Producer
Date Thu, 01 Aug 2013 04:37:15 GMT
Nope. The property "metadata.broker.list" in producer.properties [0] is
just used to fetch the metadata about which broker owns which
[topic-partition]. Based on this info, the producer data is routed to the
relevant leader broker.

By default Kafka uses DefaultPartitioner [1] which is used to randomly
assign [topic-partitions] amongst the available brokers. Once a broker is
fixed as a leader for a [topic-partition], all messages for that
[topic-partition] will end up to that fixed broker (until that broker dies,
re-balancing occurs or you run the re-assign partitions tool). I believe
that this cannot be changed via any config.

[0] : https://github.com/apache/kafka/blob/0.8/config/producer.properties
[1] :
https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/producer/DefaultPartitioner.scala


On Wed, Jul 31, 2013 at 1:47 PM, Jared Wong <jaredw@mit.edu> wrote:

> Hi,
>
> I want to randomly send my messages to a random broker and a random
> partition registered in Zookeeper.
>
> If, in my producer configuration, I only specify zk.connect and I
> don't specify any broker.list or other information, will this happen?
>
> Best,
> Jared
> --
> Jared Wong :: jaredw@mit.edu
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message