kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip O'Toole" <phi...@loggly.com>
Subject New error on on 0.72 Kafka brokers
Date Fri, 01 Nov 2013 02:18:12 GMT
We suddenly started seeing these messages from one our consumers tonight.
What do they mean? Our high-level consumers -- most of them -- are not
moving.

What has happened?

Philip

2013-11-01 01:58:09,033 [ERROR] [FetcherRunnable.error] error in
FetcherRunnable for pro
cessed:6-6: fetched offset = 403315202733: consumed offset = 403315202733
kafka.common.InvalidMessageSizeException: invalid message size: 1129552
only received by
tes: 1048572 at 403315202733( possible causes (1) a single message larger
than the fetch
 size; (2) log corruption )
        at
kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(ByteBufferMessageSet
.scala:120)
        at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scal
a:160)
        at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scal
a:100)
        at
kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59)
        at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51)
        at
kafka.message.ByteBufferMessageSet.shallowValidBytes(ByteBufferMessageSet.sca
la:64)
        at
kafka.message.ByteBufferMessageSet.validBytes(ByteBufferMessageSet.scala:59)
        at
kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:57)
        at
kafka.consumer.FetcherRunnable$$anonfun$run$6.apply(FetcherRunnable.scala:79)
        at
kafka.consumer.FetcherRunnable$$anonfun$run$6.apply(FetcherRunnable.scala:65)
        at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59
)
        at scala.collection.immutable.List.foreach(List.scala:45)
        at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:65)
2013-11-01 01:58:09,033 [DEBUG] [PartitionTopicInfo.debug] reset consume
offset of proce
ssed:6-7: fetched offset = 404145188795: consumed offset = 404143227662 to
404143227662
2013-11-01 01:58:09,034 [ERROR] [FetcherRunnable.error] error in
FetcherRunnable
kafka.common.InvalidMessageSizeException: invalid message size: 1129552
only received by
tes: 1048572 at 403315202733( possible causes (1) a single message larger
than the fetch
 size; (2) log corruption )
        at
kafka.message.ByteBufferMessageSet$$anon$1.makeNextOuter(ByteBufferMessageSet
.scala:120)
        at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scal
a:160)
        at
kafka.message.ByteBufferMessageSet$$anon$1.makeNext(ByteBufferMessageSet.scal
a:100)
        at
kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:59)
        at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:51)
        at
kafka.message.ByteBufferMessageSet.shallowValidBytes(ByteBufferMessageSet.sca
la:64)
        at
kafka.message.ByteBufferMessageSet.validBytes(ByteBufferMessageSet.scala:59)
        at
kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:57)
        at
kafka.consumer.FetcherRunnable$$anonfun$run$6.apply(FetcherRunnable.scala:79)
        at
kafka.consumer.FetcherRunnable$$anonfun$run$6.apply(FetcherRunnable.scala:65)
        at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59
)
        at scala.collection.immutable.List.foreach(List.scala:45)
        at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:65)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message