kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam William <sa...@stumbleupon.com>
Subject Re: High level consumer Question
Date Tue, 12 Jun 2012 03:05:19 GMT
Thanks Jay.. I had made such an assumption and seems like it was valid.

Sam
On Jun 11, 2012, at 7:52 PM, Jay Kreps wrote:

> Hi Sam,
> 
> We assign partitions to consumer threads so that each partition has a
> consumer. If you have more partitions then threads, some threads will get
> multiple partitions (nothing wrong with this); if you have more threads
> than partitions some thread will not get any partitions and will be idle.
> But never will one partition go to more than one thread (since the
> consumption is meant to be ordered within that partition, splitting it
> would likely break this).
> 
> -Jay
> 
> On Mon, Jun 11, 2012 at 7:11 PM, Sam William <sampd@stumbleupon.com> wrote:
> 
>> Hi,
>> Im trying out the high level consumer using the  ConsumerConnector API .
>>  I have a topic with 100 partitions  .  What does it mean if I create
>> messagestreams with
>> 
>> 
>> consumerConnector.createMessageStreams(Predef.Map(topic -> partitions),
>> decoder)
>> 
>> where partitions < 100  (say 50).  does the consumer read messages only
>> from 50 of the 100 partitions ?
>> 
>> 
>> 
>> 
>> 
>> Sam William
>> sampd@stumbleupon.com
>> 
>> 
>> 
>> 

Sam William
sampd@stumbleupon.com




Mime
View raw message