kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "S. Zhou" <myx...@yahoo.com.INVALID>
Subject Re: A wired producer connection timeout issue
Date Fri, 08 Aug 2014 21:42:55 GMT
Thanks Guozhang. Any ideas on what could be wrong on that machine? We set up multiple producers
in the same way but only one has this issue.

On Friday, August 8, 2014 2:41 PM, Guozhang Wang <wangguoz@gmail.com> wrote:

This might be due to some issue on that producer machine, the "producer queue full and message
sent rate low" is likely to be the result of the frequent connection timeout, but not the
cause of it.


On Fri, Aug 8, 2014 at 2:30 PM, S. Zhou <myxjtu@yahoo.com.invalid> wrote:

A Kafka producer frequently timeout when connecting to a remote Kafka cluster while producers
on other machine (same data center) can connect to the Kafka cluster with no problem.  From
the monitoring,  the ProductQueueSize is always full and message sent rate is low. We use
Kafka 0.8. We set "batch.num.messages=10000" and "queue.buffering.max.ms=5000". 
>Here is the error message:
>[2014-08-08 17:52:02,786] ProducerSendThread producer.SyncProducer ERROR Producer connection
to kafka-XXX.com:9092 unsuccessful
>java.net.ConnectException: Connection timed out
>        at sun.nio.ch.Net.connect(Native Method)
>        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:525)
>        at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
>        at kafka.producer.SyncProducer.connect(SyncProducer.scala:146)
>        at kafka.producer.SyncProducer.getOrMakeConnection(SyncProducer.scala:161)
>        at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:68)
>        at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
>        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
>        at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
>        at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:67)
>        at kafka.utils.Utils$.swallow(Utils.scala:187)
>        at kafka.utils.Logging$class.swallowError(Logging.scala:105)
>        at kafka.utils.Utils$.swallowError(Utils.scala:46)
>        at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:67)
>        at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
>        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
>        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
>        at scala.collection.immutable.Stream.foreach(Stream.scala:548)
>        at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
>        at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)


-- Guozhang
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message