kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laxmi Narayan <nit.dgp...@gmail.com>
Subject Re: how to listen each partition with separate threads ?
Date Fri, 31 Mar 2017 12:02:41 GMT
Hi,

Is there any performance downside of creating so many consumers ?

I mean literally I am gonna create atleast 7k connections in that case , I
have nearly 7k partitions  with a given topic.


Keep learning keep moving .....

On Fri, Mar 31, 2017 at 12:48 PM, Matthias J. Sax <matthias@confluent.io>
wrote:

> You need to create a KafkaConsumer per thread.
>
> -Matthias
>
> On 3/30/17 10:51 PM, Laxmi Narayan wrote:
> > Hi ,
> >
> > I was thinking to listen each partition with separate thread in Kafka.
> > But i get error saying :
> >
> >
> >
> >
> > *org.apache.kafka.clients.consumer.KafkaConsumer@383ad023KafkaConsumer
> is
> > not safe for multi-threaded accessSubscription to topics, partitions and
> > pattern are mutually exclusive.*​
> >
> >  Is there any way to achieve this ?
> >
> >
> > *​​*
> > Keep learning keep moving .....
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message