kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: potential bug with offset request and just rolled log segment
Date Tue, 02 Jun 2015 03:50:05 GMT
Hi Alfred,

I check the 0.8.2 code and may probably find a bug related to your issue.
Basically, segsArray.last.size is called multiple times during handling
offset requests, while segsArray.last could get concurrent appends. Hence
it is possible that in line 461, if(segsArray.last.size > 0) returns false
while later in line 468, if(segsArray.last.size > 0) could return true.

Could you file a jira for this? We can probably fix this bug in the 0.8.3
release.

As for now, if you can control your log rolling to be not too frequent this
bug will be much less likely exposed.

Guozhang


On Mon, Jun 1, 2015 at 3:14 PM, Alfred Landrum <alfred@jut.io> wrote:

> I'm running 0.8.2 in a setup that's rolling segments relatively often.
> Using my non-Kafka client that's using librdkafka, I occasionally see this
> in my kafka logs:
>
> ...
> [2015-05-30 00:30:26,149] INFO Rolled new log segment for
> 'receiver-db1fc9b7-85b2-4016-8eda-66a356cb2ae8-35' in 1 ms.
> (kafka.log.Log)
> [2015-05-30 00:30:26,152] WARN [KafkaApi-0] Error while responding to
> offset request (kafka.server.KafkaApis)
> java.lang.ArrayIndexOutOfBoundsException: 7
>         at kafka.server.KafkaApis.fetchOffsetsBefore(KafkaApis.scala:469)
>         at kafka.server.KafkaApis.fetchOffsets(KafkaApis.scala:449)
>         at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:411)
>         at kafka.server.KafkaApis$$anonfun$17.apply(KafkaApis.scala:402)
>         at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>         at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>         at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
>         at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>         at scala.collection.AbstractTraversable.map(Traversable.scala:105)
>         at kafka.server.KafkaApis.handleOffsetRequest(KafkaApis.scala:402)
>         at kafka.server.KafkaApis.handle(KafkaApis.scala:61)
>         at
> kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:59)
>         at java.lang.Thread.run(Thread.java:745)
> ...
>
>
> Does this match up with any known issues?  I didn't find matching bugs on
> issues.apache.org.
>



-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message