kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bateman, Matt" <mabate...@ebay.com>
Subject RE: Only one broker has partitions
Date Fri, 17 Feb 2012 21:40:08 GMT
Hi Jun,

Thanks for the details. What sort of time frame is eventually?

In the case of a topic that exists before nodes are added, will the other nodes ever get that
topic? We've been running all three for about a week now it's still only on our 0 broker.

Thanks,

Matt

-----Original Message-----
From: Jun Rao [mailto:junrao@gmail.com] 
Sent: Friday, February 17, 2012 11:19 AM
To: kafka-users@incubator.apache.org
Subject: Re: Only one broker has partitions

Matt,

The behavior is a bit weird for a new topic. Initially, non of the brokers has the topic.
So, the producer simply assumes there is 1 partition per broker. Once a topic is created in
a broker, the broker registers #partitions for that topic in ZK. Then the producer will pick
up the new #partitions for that broker and refresh its list of available partitions.
Eventually, the topic will be created on every broker and the producer should see a total
of 60 partitions.

Thanks,

Jun


On Fri, Feb 17, 2012 at 11:05 AM, Bateman, Matt <mabateman@ebay.com> wrote:

> We're using a single topic ("test-topic") in our QA setup. Originally 
> we were using only one broker (id 0) in or testing environment. 
> Recently we added two more (id 2 and 3) for a total of 3 in our 
> cluster. ZooKeeper shows that the topic and partitions are registered by broker 0 as
expected.
> They do not show for brokers 2 and 3. I'm guessing that if a broker is 
> not registered with ZooKeeper when a producer sends the first message 
> for a topic, it will never process messages for that topic?
>
> As a test, I sent messages into the cluster for a new topic. The 
> producer logs show
>
> [2012-02-17 10:31:12,698] DEBUG Getting the number of broker 
> partitions registered for topic: cluster-test 
> (kafka.producer.Producer)
> [2012-02-17 10:31:12,698] DEBUG Currently, no brokers are registered 
> under
> topic: cluster-test (kafka.producer.ZKBrokerPartitionInfo)
> [2012-02-17 10:31:12,698] DEBUG Bootstrapping topic: cluster-test with 
> available brokers in the cluster with default number of partitions = 1
> (kafka.producer.ZKBrokerPartitionInfo)
> [2012-02-17 10:31:12,705] DEBUG Adding following broker id, partition 
> id for NEW topic: cluster-test=TreeSet(0-0, 2-0, 3-0)
> (kafka.producer.ZKBrokerPartitionInfo)
> [2012-02-17 10:31:12,705] DEBUG Broker partitions registered for topic:
> cluster-test = List(0-0, 2-0, 3-0) (kafka.producer.Producer)
>
> This seems logical, all brokers are mentioned. Then I see the 
> following (each broker is configured to use 20 partitions):
>
> [2012-02-17 10:31:12,840] DEBUG Sending message to broker 2
> (kafka.producer.ProducerPool)
> [2012-02-17 10:31:12,953] DEBUG [BrokerTopicsListener] List of topics 
> changed at /brokers/topics Updated topics -> [topic1, topic2, 
> cluster-test, topic3, test-topic, topic4]
> (kafka.producer.ZKBrokerPartitionInfo$BrokerTopicsListener)
> [2012-02-17 10:31:12,953] DEBUG [BrokerTopicsListener] Old list of topics:
> Set(topic1, topic3, topic2, test-topic, topic4)
> (kafka.producer.ZKBrokerPartitionInfo$BrokerTopicsListener)
> [2012-02-17 10:31:12,963] DEBUG [BrokerTopicsListener] Updated list of
> topics: Set(topic4, test-topic, topic2, topic3, cluster-test, topic1)
> (kafka.producer.ZKBrokerPartitionInfo$BrokerTopicsListener)
> [2012-02-17 10:31:12,963] DEBUG [BrokerTopicsListener] List of newly 
> registered topics: Set(cluster-test)
> (kafka.producer.ZKBrokerPartitionInfo$BrokerTopicsListener)
> [2012-02-17 10:31:12,978] DEBUG [BrokerTopicsListener] Currently 
> registered list of brokers for topic: cluster-test are Buffer(2)
> (kafka.producer.ZKBrokerPartitionInfo$BrokerTopicsListener)
> [2012-02-17 10:31:12,978] DEBUG [BrokerTopicsListener] Unregistered 
> list of brokers for topic: cluster-test are TreeSet(0-0, 2-0, 3-0)
> (kafka.producer.ZKBrokerPartitionInfo$BrokerTopicsListener)
> [2012-02-17 10:31:12,980] DEBUG [BrokerTopicsListener] List of broker 
> partitions for topic: cluster-test are TreeSet(0-0, 2-0, 2-1, 2-2, 
> 2-3, 2-4, 2-5, 2-6, 2-7, 2-8, 2-9, 2-10, 2-11, 2-12, 2-13, 2-14, 2-15, 
> 2-16, 2-17, 2-18, 2-19, 3-0)
> (kafka.producer.ZKBrokerPartitionInfo$BrokerTopicsListener)
> [2012-02-17 10:31:13,841] DEBUG Getting the number of broker 
> partitions registered for topic: cluster-test 
> (kafka.producer.Producer)
> [2012-02-17 10:31:13,841] DEBUG Broker partitions registered for topic:
> cluster-test = List(0-0, 2-0, 2-1, 2-2, 2-3, 2-4, 2-5, 2-6, 2-7, 2-8, 
> 2-9, 2-10, 2-11, 2-12, 2-13, 2-14, 2-15, 2-16, 2-17, 2-18, 2-19, 3-0)
> (kafka.producer.Producer)
>
> This seems strange, shouldn't each broker host each partition for the 
> new "cluster-topic"?
>
> I then see the producer debug messages showing messages sent to broker 2.
>
> What am I missing here?
>
> Thanks,
>
> Matt
>
> -----Original Message-----
> From: Jun Rao [mailto:junrao@gmail.com]
> Sent: Thursday, February 16, 2012 6:08 PM
> To: kafka-users@incubator.apache.org
> Subject: Re: Only one broker has partitions
>
> Try enabling debug level logging in Producer class. It will show which 
> broker the partitioner selects.
>
> Thanks,
>
> Jun
>
> On Thu, Feb 16, 2012 at 2:48 PM, Bateman, Matt <mabateman@ebay.com> wrote:
>
> > Hi All,
> >
> > I feel like this was discussed at some point but couldn't find it in 
> > the docs or the mailing list archives, hopefully this isn't redundant.
> >
> > We are running 3 version 0.7 brokers using an ensemble of 3 
> > ZooKeeper nodes. We use a ZooKeeper connect string in the producers and consumers.
> > ZooKeeper is enabled on each broker. We use the default partitioner 
> > in the producer supplying it an integer. Each broker has a unique Id.
> >
> > However, only one broker has any log files. The log from that 
> > machine shows it registering the topics in ZooKeeper. The other 
> > brokers don't have any log files.
> >
> > Have a I missed a basic configuration setting or misunderstood 
> > something operational? The only slight mismatch I see is we run 
> > ZooKeeper 3.3.4 but it seems like 3.3.3 is being used by 0.7.
> >
> > Thanks,
> >
> > Matt
> >
>

Mime
View raw message