kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tarang Dawer <tarang.da...@gmail.com>
Subject Consumer not able to create message streams
Date Wed, 04 Dec 2013 12:10:51 GMT
Hi All

I am using Kafka 0.8 , with inbuilt zookeeper.
I am runnings consumers in a jar.



Configuration properties for creating consumerConfig : -

zookeeper.connect=IP
group.id=consumerGroup
fetch.message.max.bytes=1000000000
zookeeper.session.timeout.ms=60000
auto.offset.reset=smallest
zookeeper.sync.time.ms=200
auto.commit.enable=false

While running it, some times the consumer gets stuck on the message stream
creation call i.e


ConsumerConfig consumerConfig = new ConsumerConfig(this.props);

            ConsumerConnector consumerConnector = Consumer
                    .createJavaConsumerConnector(consumerConfig);


*Map<String, List<KafkaStream<byte[], byte[]>>> messageStreamMap =
consumerConnector                    .createMessageStreams(topicMap)*;

I have tested this thing on a number of partition settings , i.e 1,2,5 etc.

Whenever the consumer gets stuck, i have to kill the jar, and start it
again. This thing happens only a select couple of times , i.e 2 or 3 times
out of 10, and for rest of the times, the consumer starts fine.

Is there some property configuration that i may have been missing, that's
causing this inconsistent behaviour ?


Any guidance would be of great help.

Thanks
Tarang Dawer

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message