kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: ways to exit the blocking iterator
Date Tue, 28 Feb 2012 17:43:57 GMT
>> should ideally deliver a unblock message to  the iterator and then continue
delivering the messages to the queue ; and by implication; to the consumer
iterator (of course, in 0.7, also recreating the MessageStream due to the
timeout issue)

I'm not sure what you mean by this. What will happen, in your example,
is that the consumer iterator will get messages 1 and 2 only.
If you have multiple instances of the zookeeper consumer, they will
get messages 3 and 4. If you don't, then you will get 3 and 4, the
next time you start the consumer connector again.

Thanks,
Neha

On Tue, Feb 28, 2012 at 6:08 AM, Joe Stein <cryptcom@gmail.com> wrote:
> Milind, if you are using the ZookeeperConsumerConnector then you can just
> call shutdown() on it.
>
> This will internally call sendShutdownToAllQueues which will pass the
> "poison pill " of ZookeeperConsumerConnector.shutdownCommand to all queues.
>
> On Tue, Feb 28, 2012 at 8:29 AM, Milind Parikh <milindparikh@gmail.com>wrote:
>
>> I need to exit the blocking consumer iterator on certain conditions ; do
>> some other processing and then resume the blocking iterartor. All of the
>> work is dependent on the "other processing".
>>
>> Upon looking at the code, it seems that
>>
>> https://svn.apache.org/repos/asf/incubator/kafka/trunk/core/src/main/scala/kafka/consumer/ConsumerIterator.scala
>> can take a shutdown command that will trigger the iterator to come out of
>> the blocking state.
>>
>> Two Questions....
>> (a) Is there some sample code  that will issue this shutdown command?
>> (b) Is there any implication to the semantics of the queue that this
>> iterator is blocking on ?
>>
>> (for example)
>>
>> Message_1
>> Message_2
>> SHUTDOWN
>> Message_3
>> Message_4
>>
>> should ideally deliver a unblock message to  the iterator and then continue
>> delivering the messages to the queue ; and by implication; to the consumer
>> iterator (of course, in 0.7, also recreating the MessageStream due to the
>> timeout issue)
>>
>> Regards
>> Milind
>>
>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */

Mime
View raw message