kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjkosh...@gmail.com>
Subject Re: Consumer group and Partitions
Date Mon, 25 Jun 2012 17:47:32 GMT
That should be all you need - although you only need to bounce the kafka
broker. Are you sending enough messages? If you send only a few test
messages, it is possible for the messages to get sent to only one
partition. E.g., I tried a quick test with the ConsoleProducer and it took
three messages before the partition was created.

Joel

On Mon, Jun 25, 2012 at 8:43 AM, Narala, Ravi <Ravi.Narala@level3.com>wrote:

> Jun ,
> Thanks for the quick repsonse. I am running into a partition issue. I am
> not sure if am doing the right way. Here are the steps I did to create the
> partition. But I still do not see them partitioned.
>
> Stopped the kafka and zookeeper server
> Modified the num.partitions entry in server.properties from 1 to 2.
> Restarted the zookeeper and kafka server
> Used a Producer client to send messages to a topic named
> 'poc_test_topic1'.(I did not specify any partition key)
>
> The logs dir was configured to /tmp/kafka-logs in server.properties.
>
> I see only a 'poc_test_topic1-0' topic under logs dir. I was expecting to
> see 'poc_test_topic1-0' and 'poc_test_topic1-1' partitions under it.
>
> Am I missing something or is this all I need to create a partition ?
>
> Ravi Narala
> Web Services & Integration
> Level 3 Communications
> 200 Office Galleria Center
> Southfield, MI 48035 US
> p: 248-436-9324
> e: ravi.narala@level3.com
>
> -----Original Message-----
> From: Jun Rao [mailto:junrao@gmail.com]
> Sent: Monday, June 25, 2012 11:05 AM
> To: kafka-users@incubator.apache.org
> Subject: Re: Consumer group and Partitions
>
> Ravi,
>
> For 1, consumer group is specified in consumer config.
>
> For 2, partitions are divided evenly among consumers in the same group for
> consumption. For details, please see the design doc on our website.
>
> For 3, a producer never needs to know consumer groups. A producer always
> produces messages to a topic/partition. There are examples in the design
> doc too.
>
> Thanks,
>
> Jun
>
> On Mon, Jun 25, 2012 at 7:57 AM, Narala, Ravi <Ravi.Narala@level3.com
> >wrote:
>
> > Hi,
> > I have couple of questions regarding the concepts of Consumer group
> > and partitions. Here are those ?
> >
> >
> > 1.       When consuming a message how Do I know which consumer group I
> > belong to?. Is there a configuration available or do we have to
> > specify it at the application level ?
> >
> > 2.       How do consumer groups work in conjunction with partitions ?
> >
> > 3.       How do I produce a message to specific group and a partition ?.
> A
> > coding example is greatly appreciated.
> >
> >
> >
> > Thanks,
> >
> > Ravi Narala
> > Web Services & Integration
> > Level 3 Communications
> > 200 Office Galleria Center
> > Southfield, MI 48035 US
> > p: 248-436-9324
> > e: ravi.narala@level3.com<mailto:ravi.narala@level3.com>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message