kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cosmin Marginean <cosmargin...@gmail.com>
Subject Bizarre crash when creating a consumer
Date Thu, 07 Jan 2016 22:23:19 GMT
Hi  

I have a straightforward piece of code that creates a consumer (Kafka 0.9.0.0).

Properties props = new Properties();
props.put("zookeeper.connect", zookeeperServers);
props.put(org.apache.kafka.clients.consumer.ConsumerConfig.GROUP_ID_CONFIG, groupId);
log.info("Starting consumer group for topic {} and group ID {}. Zookeeper servers: {}", topic,
groupId, zookeeperServers);
consumer = kafka.consumer.Consumer.createJavaConsumerConnector(new ConsumerConfig(props));
log.info("Consumer group started for topic {} and group ID {}", topic, groupId);
We’ve run this countless times without any issues, but now we’re deploying a new environment
(AWS, just like the ones before) and it appears that the client Java process dies entirely
(without any logs/crash report/etc). This happens right after logging the “Starting consumer
group..”, so presumably when it tries to createJavaConsumerConnector
Agreeably, this might be “environmental”, but even though we triple checked everything
(network setup, kafka logs, zookeeper logs, etc), we couldn’t identify anything suspicious
yet. So what I'd like to know is if there’s a way to add further Kafka diagnosis/logging.
Attached (trace-logging.txt) is further logging after turning everything to TRACE, and at
the top you can see the message “Starting consumer…”, but with nothing really suspicious
as far as I can tell.

As an additional piece of information, Zookeeper does report the following when this happens
2016-01-07 21:58:44,763 [myid:1] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357]
- caught end of stream exception EndOfStreamException: Unable to read additional data from
client sessionid 0x1521e14797c0001, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) at
java.lang.Thread.run(Thread.java:745) 2016-01-07 21:58:44,764 [myid:1] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007]
- Closed socket connection for client /10.100.101.159:41613 which had sessionid 0x1521e14797c0001

Any suggestions would be appreciated.
Thank you
Cosmin


Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message