kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Corbin <DCor...@lancope.com>
Subject Re: Problems getting offsets
Date Sun, 01 Nov 2015 20:38:34 GMT
Yes.  I know all of that.  I guess my original message was not clear.

The topic metadata indicates that there is a leader.

Please see my comments interspersed below.
Thanks
David Corbin

On 10/29/15, 17:53, "Mayuresh Gharat" <gharatmayuresh15@gmail.com> wrote:

> NotCoordinatorForConsumerException is for consumer side.

I am attempting to consume messages, so this means I¹m getting an
exception that might be reasonable for what I¹m doing.

>I think if you
>are using simpleConsumer, then you have to do your own offset management.

I am attempting to manage my offsets myself; that¹s why I called
SimpleConsumer#fetchOffsets() which is throwing the exception.  I assume
you are not suggesting that my offset management should be done entirely
³outside the scope² of Kafka.  If that is true, why do 4 of 3 methods on
SimpleConsumer refer to offsets?


>
>TopicMetadata tells you whether there is a leader for the topic
>partitions,
>replicas, ISR.


Yes.  After I create the topic, I ³poll² the metadata until the one
topic/partition has a leader, before moving on.  If the metadata says
there is a leader, I don¹t understand whyI would get
NotCoordinatorForConsumerException

>
>Thanks,
>
>Mayuresh
>
>On Wed, Oct 28, 2015 at 8:23 AM, David Corbin <DCorbin@lancope.com> wrote:
>
>> I'm working on a project that I hope to use the SimpleConsumer on.  I'm
>> trying to write some test code around our code that is wrapping the
>> SimpleConsumer.
>>
>> The environment is 1 kafka broker, and 1 zookeeper
>> The test goes like this:
>>
>> Create a topic ("foo", 1 partition, 1 replica)
>> Create a Producer, and send a message
>> Create a SimpleConsumer, and try to read the offset
>>
>> Failure with: NotCoordinatorForConsumerException
>>
>> If I continue to require for an extended period, I continue to get that
>> exception.
>>
>> As near as I can tell, the topic metadata says there is a leader, but
>>the
>> the broker thinks it's being rebalanced.
>> I've done the above test immediately after stop, clean out old data, and
>> restart both zookeeper and kafka.
>>
>> Suggestions welcome.
>>
>>
>
>
>-- 
>-Regards,
>Mayuresh R. Gharat
>(862) 250-7125


Mime
View raw message