kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arjun <ar...@socialtwist.com>
Subject offset asked for broker is more than what broker has when does this happen - High level consumer -kafka 0.8
Date Sun, 16 Feb 2014 05:06:52 GMT
Hi,

I have set up a clean kafka set up on three ec2 nodes. I have pushed in 
two messages into the set up. But i invoke the high level consumer, i 
get this error in the kafka broker. I ran the Consumer offset tool. It 
says the lag to be -6. This set up is fresh set up and it has three 
kafka brokers and three zookeepers running, with replication factor 2.

Can some one please let me know when will this arise? and what is work 
around for this?



2014-02-15 23:50:10,760] ERROR [KafkaApi-0] Error when processing fetch 
request for partition [taf.referral.emails.service,1] offset 10 from 
consumer with correlation id 34 (kafka.server.KafkaApis)
kafka.common.OffsetOutOfRangeException: Request for offset 10 but we 
only have log segments in the range 0 to 4.
     at kafka.log.Log.read(Log.scala:429)
     at 
kafka.server.KafkaApis.kafka$server$KafkaApis$$readMessageSet(KafkaApis.scala:388)
     at 
kafka.server.KafkaApis$$anonfun$kafka$server$KafkaApis$$readMessageSets$1.apply(KafkaApis.scala:334)
     at 
kafka.server.KafkaApis$$anonfun$kafka$server$KafkaApis$$readMessageSets$1.apply(KafkaApis.scala:330)
     at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
     at 
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
     at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
     at 
scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
     at scala.collection.immutable.Map$Map1.map(Map.scala:93)
     at 
kafka.server.KafkaApis.kafka$server$KafkaApis$$readMessageSets(KafkaApis.scala:330)
     at kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:296)
     at kafka.server.KafkaApis.handle(KafkaApis.scala:66)
     at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42)
     at java.lang.Thread.run(Thread.java:662)
^C
ubuntu@ip-10-235-39-219:~/kafka/kafka_2.8.0-0.8.0$ ^C
ubuntu@ip-10-235-39-219:~/kafka/kafka_2.8.0-0.8.0$ 
bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group group1 
--zkconnect 10.235.39.219:2181 --topic taf.referral.emails.service
Group           Topic                          Pid Offset 
logSize         Lag             Owner
group1          taf.referral.emails.service    0   10 4               -6 
group1_ec2-54-225-44-248.compute-1.amazonaws.com-1392525076823-fb0973d5-0
group1          taf.referral.emails.service    1   10 4               -6 
group1_ec2-54-225-44-248.compute-1.amazonaws.com-1392525076823-fb0973d5-1


Thanks
Arjun Narasimha Kota

Mime
View raw message