kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aniket Kulkarni <kulkarnianiket...@gmail.com>
Subject Connection reset by peer
Date Thu, 25 Sep 2014 18:57:21 GMT
Hello Neha,

I am trying to run some tests which use Kafka The tests do not
fail but give out a warning messages which I am trying to get rid off such
as :

2014-09-25 11:43:03,572 [kafka-processor-56598-1] ERROR
kafka.network.Processor - Closing socket for / because of error

java.io.IOException: Broken pipe

at sun.nio.ch.FileDispatcherImpl.write0(Native Method)

at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)

at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)

at sun.nio.ch.IOUtil.write(IOUtil.java:65)

at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:487)

at kafka.api.PartitionDataSend.writeTo(FetchResponse.scala:67)

at kafka.network.MultiSend.writeTo(Transmission.scala:101)

at kafka.api.TopicDataSend.writeTo(FetchResponse.scala:124)

at kafka.network.MultiSend.writeTo(Transmission.scala:101)

at kafka.api.FetchResponseSend.writeTo(FetchResponse.scala:219)

at kafka.network.Processor.write(SocketServer.scala:375)

at kafka.network.Processor.run(SocketServer.scala:247)

at java.lang.Thread.run(Thread.java:744)


2014-09-25 11:43:53,770 [kafka-processor-56598-1] ERROR
kafka.network.Processor - Closing socket for / because of error

java.io.IOException: Connection reset by peer

at sun.nio.ch.FileDispatcherImpl.read0(Native Method)

at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)

at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)

at sun.nio.ch.IOUtil.read(IOUtil.java:197)

at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)

at kafka.utils.Utils$.read(Utils.scala:375)


at kafka.network.Processor.read(SocketServer.scala:347)

at kafka.network.Processor.run(SocketServer.scala:245)

 at java.lang.Thread.run(Thread.java:744)

I tried to debug the producer and consumer I am using but it turns out this
exception is thrown when I try to close the producer or consumer. I tried a
couple of approaches as mentioned in a few threads I saw online :

1.) I tried to increase the fetch.wait.max.ms and socket.timeout.ms for the
consumer but that didn't work

2.) A couple of threads suggested firewall issues. Since I am running the
tests locally, firewall issues are irrelevant since the broker is not

3.) I tried to catch the IOException in the tests which threw the above
mentioned exceptions but I was not able to catch them. This led me to
believe that the exceptions are not occurring on the producer and/or
consumer but on the broker. I think the broker seems to be closing open or
stray connections, which are giving out those exceptions. I also confirmed
that all my producer and consumer tests are closing connections to the
broker as soon as the test is executed.

Also, in a comment on this thread you mentioned that this is an expected
exception. Does that mean there will be a fix out for this?

Does this issue refer to the fix?

[1] - https://issues.apache.org/jira/browse/KAFKA-270

Thanks a lot for your help.

Aniket Kulkarni.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message