kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: Topic messages with partitions=1 stored on multiple brokers
Date Fri, 20 Sep 2013 15:24:07 GMT
Hello Suchi,

As Philip said in 0.7 the number of partitions is per broker. As from 0.8
the number of partitions will be global, meaning the total number of
partitions across the broker cluster.

Guozhang


On Fri, Sep 20, 2013 at 3:50 AM, Philip O'Toole <philip@loggly.com> wrote:

> Seems to me you are confusing partitions and brokers. Partition count has
> nothing to do with the number of brokers to which a message a sent -- just
> the number of partitions into which that message will be split when it gets
> to a broker.
>
> You need to explicitly set the destination brokers in the Producer, not via
> setting "partition" count.
>
> Philip
>
>
> On Fri, Sep 20, 2013 at 1:53 AM, Suchi Amalapurapu <suchi@bloomreach.com
> >wrote:
>
> > My current setup has 2 brokers, multiple topics, each topic with 1
> > partition, kafka 0.7.1
> > Some topics are logged onto multiple brokers inspite of partitions set to
> > 1. Kafka 0.7.1 does not have replication turned on.
> > Why are the messages being logged onto multiple brokers when the number
> of
> > partitions is set to 1?
> > I have a consumer with parallelism set to 1 and its missing some of these
> > messages.
> > Suchi
> >
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message