kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Consumer in the mirror maker stop fetching logs
Date Fri, 03 May 2013 15:50:14 GMT
The consumer hit the exception because the broker closed the socket. What
does the broker log around the same time say? It should tell you the reason
why the broker closed the socket.

Thanks,

Jun


On Fri, May 3, 2013 at 2:34 AM, Helin Xiang <xkeegan@gmail.com> wrote:

> hi,
>
> We are doing some mirror-maker test。We currently use version 0.7.2。
>
> some times,the mirror maker will throw exceptions like this, and then
> stopped fetching topics:
>
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
>      at kafka.utils.Utils$.read(Utils.scala:539)
>      at
>
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:67)
>      at kafka.network.Receive$class.readCompletely(Transmission.scala:55)
>      at
>
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
>      at kafka.consumer.SimpleConsumer.getResponse(SimpleConsumer.scala:177)
>      at
> kafka.consumer.SimpleConsumer.liftedTree2$1(SimpleConsumer.scala:125)
>      at kafka.consumer.SimpleConsumer.multifetch(SimpleConsumer.scala:115)
>      at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:60)
> [2013-05-02 15:13:05,339] INFO stopping fetcher FetchRunnable-0 to host
> kf.a.com (kafka.consumer.FetcherRunnable)
>
> I used jconsole to double check, and confirmed that the consumer in the
> mirror maker stopped fetching logs brokers。I have to stop the mirror maker
> and start it over, and it continued fetching logs。
>
> I start consumers using only 1 consumer thread。
>
> By the way,there are 16 mirror makers in the consume group。
> I did some calculation and it can be assured that the partition numbers are
> larger than or equal to the consumer threads。
>
> Thats all I got。So, I wonder if there was a bug issue for just mirror maker
> or any applications using the high level consumers will behave like this?
>
>
> thanks!
>
>
> --
> *Best Regards
>
> Helin Xiang*
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message