kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ankit tyagi <ankittyagi.mn...@gmail.com>
Subject Potential socket leak in kafka sync producer
Date Thu, 29 Jan 2015 11:06:09 GMT
Hi,

Currently we are using sync producer client of 0.8.1 version in our
production box . we are getting the following exception while publishing
kafka message

*[2015-01-29
13:21:45.505][ThreadPoolTaskExecutor-603][WARN][ClientUtils$:89] Fetching
topic metadata with correlation id 10808 for topics [Set(*
*kafka_topic_coms_FD_test1)] from broker [id:0,host:localhost,port:9092]
failed*
*java.net.ConnectException: Connection refused*
*        at sun.nio.ch.Net.connect0(Native Method)*
*        at sun.nio.ch.Net.connect(Net.java:465)*
*        at sun.nio.ch.Net.connect(Net.java:457)*
*        at
sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670)*
        at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
        at kafka.producer.SyncProducer.connect(SyncProducer.scala:141)
        at
kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:156)
        at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68)
        at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
        at
kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
        at
kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)


we are using dynamic thread pool to publish message to kafka. My
observation is when after keep alive time when threads in my executor gets
destroyed, somehow file descriptor is not getting cleared but when i did
explicitly ran the full gc, fd count got reduced by a signification amout.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message