kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: High level consumer Question
Date Tue, 12 Jun 2012 02:52:04 GMT
Hi Sam,

We assign partitions to consumer threads so that each partition has a
consumer. If you have more partitions then threads, some threads will get
multiple partitions (nothing wrong with this); if you have more threads
than partitions some thread will not get any partitions and will be idle.
But never will one partition go to more than one thread (since the
consumption is meant to be ordered within that partition, splitting it
would likely break this).

-Jay

On Mon, Jun 11, 2012 at 7:11 PM, Sam William <sampd@stumbleupon.com> wrote:

> Hi,
>  Im trying out the high level consumer using the  ConsumerConnector API .
>   I have a topic with 100 partitions  .  What does it mean if I create
> messagestreams with
>
>
>  consumerConnector.createMessageStreams(Predef.Map(topic -> partitions),
> decoder)
>
> where partitions < 100  (say 50).  does the consumer read messages only
> from 50 of the 100 partitions ?
>
>
>
>
>
> Sam William
> sampd@stumbleupon.com
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message