kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shashank Singh <shashank.ru...@gmail.com>
Subject kafka consumer group API
Date Thu, 09 Jul 2015 18:03:08 GMT
Hi Team

I was going over the documentation and links related to consumer API
provided in
0.8.2.1 release
I got to know of major improvement in consumer API in 0.8.3 release. May I
know when we are planning to have this release.

One more help - I was following up the example given on
https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example

In this approach we can't decide
a) how to get the current offset
b) how to set a particular consumer group to start reading from a specific
offset

Also how to use the same example to read one message at a time from a topic
with many topics. I tried with creating a list of iterators and then
checking for hasNext() but that call in blocking. Even though I am using a
timeout on that call this seems not be working.

Instead what I have settled with is an internal queue to which the
multi-threaded consumer offers the messages into. Poll API polls the
internal queue. Problem is in case client code gets down without consuming
all the messages the offset is already set and the messages which have
already been read from iterator and put in queue will be never seen again.


-- 

*Warm Regards,*

*Shashank  *

*Mobile: +91 9910478553 *

*Linkedin: in.linkedin.com/pub/shashank-singh/13/763/906/
<http://in.linkedin.com/pub/shashank-singh/13/763/906/>*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message