kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjko...@gmail.com>
Subject Re: Precision on partitions
Date Tue, 04 Oct 2011 15:24:12 GMT
Yes - any given partition is wholly on one broker and is basically the
smallest unit of parallelism with regard to producing/consuming
messages.

By default, each topic gets a single partition on the node that
receives a message from that topic - you can adjust this through
either a global setting (num.partitions) that applies to all topics or
a per-topic setting (topic.partition.count.map) in the broker config.
If you use the high-level (zk producer) and don't do any custom
partitioning, the default partitioner will effectively spread the load
and you should get one partition for that topic on each broker.

Thanks,

Joel

On Tue, Oct 4, 2011 at 5:02 AM, Mathias Herberts
<mathias.herberts@gmail.com> wrote:
> Hi,
>
> one thing I'd like to have confirmed (or not) is the fact that a
> partition is managed by a single broker.
>
> If this is indeed the case, does this mean that to take full advantage
> of additional brokers the number of partitions also needs to be
> increased?
>
> If a partition is not managed by a single broker, then how does the
> consumer know how to order the messages coming from different brokers
> but pertaining to the same partition?
>
> Sorry for my newbieness.
>
> Mathias.
>

Mime
View raw message