kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bob Jervis <bjer...@gmail.com>
Subject More SocketTimeoutExceptions
Date Fri, 29 Mar 2013 22:35:32 GMT
I now have the following settings (in various configs):

In our producer configs:

producer.request.timeout.ms=600000

This producer just hangs there for 10 minutes before timing out

Here is the stack dump for that timeout:

java.net.SocketTimeoutException
        at
sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:201)
        at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)
        at
java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:221)
        at kafka.utils.Utils$.read(Utils.scala:372)
        at
kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
        at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
        at
kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
        at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
        at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
        at
kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
        at
kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SyncProducer.scala:98)
        at
kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:98)
        at
kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:98)
        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
        at
kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$sp(SyncProducer.scala:97)
        at
kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:97)
        at
kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:97)
        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
        at kafka.producer.SyncProducer.send(SyncProducer.scala:96)
        at
kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:221)
        at
kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:91)
        at
kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$1.apply(DefaultEventHandler.scala:85)
        at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
        at
scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
        at scala.collection.Iterator$class.foreach(Iterator.scala:631)
        at
scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
        at
scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
        at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
        at
kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:85)
        at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:61)
        at kafka.producer.Producer.send(Producer.scala:76)
        at kafka.javaapi.producer.Producer.send(Producer.scala:41)
        at
com.visibletechnologies.platform.common.kafka.KafkaWriter.flush(KafkaWriter.java:131)
        at
com.visibletechnologies.platform.ingestion.ContentWriter.flushToKafka(ContentWriter.java:394)
        at
com.visibletechnologies.platform.ingestion.Midas.processPosts(Midas.java:430)
        at
com.visibletechnologies.platform.ingestion.Midas.doWork(Midas.java:194)
        at
com.visibletechnologies.framework.servicebase.ServiceBase.start(ServiceBase.java:187)
        at
com.visibletechnologies.platform.ingestion.Main.main(Main.java:413)
2013-03-29 22:31:58,633 INFO kafka.client.ClientUtils$: Fetching metadata
for topic Set(VTFull-enriched)


Kafka server.properties (both brokers):

replica.socket.timeout.ms=200000
controller.socket.timeout.ms=200000
zk.connectiontimeout.ms=1000000

After seeing these issues last week, the problems magically vanished, but
yesterday they came back and we are now seeing consistent
SocketTimeoutExceptions both in the producers and in the brokers

We never see these issues where we have only one broker running.

Thanks,
Bob Jervis
Visible Technologies

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message