kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marko Bonaći <marko.bon...@sematext.com>
Subject Re: Bizarre crash when creating a consumer
Date Thu, 07 Jan 2016 22:55:46 GMT
Hi Cosmin,
do you have default server configuration on these new nodes you're setting
up?
I'd check consumer's socket.timeout.ms, maybe someone set it to 30 instead
of 30 000 :)
Speaking from my own experience (I had the same symptom and this turned out
to be the cause).

Marko Bonaći
Monitoring | Alerting | Anomaly Detection | Centralized Log Management
Solr & Elasticsearch Support
Sematext <http://sematext.com/> | Contact
<http://sematext.com/about/contact.html>

On Thu, Jan 7, 2016 at 11:23 PM, Cosmin Marginean <cosmarginean@gmail.com>
wrote:

> Hi
>
> I have a straightforward piece of code that creates a consumer (Kafka
> 0.9.0.0).
>
> Properties props = new Properties();
> props.put("zookeeper.connect", zookeeperServers);
> props.put(org.apache.kafka.clients.consumer.ConsumerConfig.GROUP_ID_CONFIG, groupId);
> log.info("Starting consumer group for topic {} and group ID {}. Zookeeper servers: {}",
topic, groupId, zookeeperServers);
> consumer = kafka.consumer.Consumer.createJavaConsumerConnector(new ConsumerConfig(props));
> log.info("Consumer group started for topic {} and group ID {}", topic, groupId);
>
> We’ve run this countless times without any issues, but now we’re deploying a new
environment (AWS, just like the ones before) and it appears that the client Java process dies
entirely (without any logs/crash report/etc). This happens right after logging the “Starting
consumer group..”, so presumably when it tries to createJavaConsumerConnector
>
> Agreeably, this might be “environmental”, but even though we triple checked everything
(network setup, kafka logs, zookeeper logs, etc), we couldn’t identify anything suspicious
yet. So what I'd like to know is if there’s a way to add further Kafka diagnosis/logging.
Attached (trace-logging.txt) is further logging after turning everything to TRACE, and at
the top you can see the message “Starting consumer…”, but with nothing really suspicious
as far as I can tell.
>
>
> As an additional piece of information, Zookeeper does report the following when this
happens
>
> 2016-01-07 21:58:44,763 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357]
- caught end of stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 0x1521e14797c0001,
likely client has closed socket
> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> 	at java.lang.Thread.run(Thread.java:745)
> 2016-01-07 21:58:44,764 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1007]
- Closed socket connection for client /10.100.101.159:41613 which had sessionid 0x1521e14797c0001
>
>
> Any suggestions would be appreciated.
>
> Thank you
>
> Cosmin
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message