kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bateman, Matt" <mabate...@ebay.com>
Subject Managing KafkaMessageStreams
Date Fri, 30 Sep 2011 00:14:10 GMT
Hi All,

Apologies if I've missed something obvious.

First, is there any reason not to call the ConsumerConnector.createMessageStreams() more than
once to create streams for the same topic? Or should all streams be created at a single point
in time?

Second, is there a "safe" way to stop reading from a KafkaMessageStream? Currently I'm just
interrupting the thread when I need to stop reading from the stream. There is no "close" or
"shutdown" method though. (Also, the Scala class doesn't annotate that InterruptedException
can be thrown, requiring some gymnastics to be performed into to correctly handle that exception.)

Third, (depending on the answer to 2) is it better to create more than one ConsumerConnector
to manage sets of streams? The ConsumerConnector seems to have a clear "shutdown" method.
Calling that would shut down all the streams that are associated with it.

Fourth, I see the "consumerTimeoutMs" attribute to timeout a consumer. Is there a cleaner
way to achieve this at a more granular level? For instance, providing a timeout duration in
the call to ConsumerIterator.next() that will simply return null?

Thanks,

Matt

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message