kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Seshadri, Balaji" <Balaji.Sesha...@dish.com>
Subject RE: Single thread, Multiple partitions
Date Tue, 08 Apr 2014 18:08:38 GMT
I think you are looking for accessing messages from set of partitions by your own policy.You
should use simple consumers in 0.8 and maintain the offsets you have read.

https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example

If it is 0.9 I'm yet to come upto speed.

Thanks,

Balaji


-----Original Message-----
From: Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) [mailto:skadambi@bloomberg.net] 
Sent: Tuesday, April 08, 2014 11:58 AM
To: users@kafka.apache.org
Subject: Single thread, Multiple partitions

Let's say I've a single consumer thread reading off multiple partitions (I'll have around
10K partitions). As per the documentation on https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example,
there are no guarantees on the order in which messages are read off the set of partitions.
If I wanted to enforce priority-weighted round robin reads off the partitions, could I get
a pointer on what code to fiddle with? Thanks!

Mime
View raw message