kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <...@confluent.io>
Subject Re: KafkaException: Should not set log end offset on partition
Date Thu, 04 Dec 2014 00:37:06 GMT
Could you look at the broker registration in Zookeeper (
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper)
and make sure the 3 registered hosts are unique?

Thanks,

Jun

On Wed, Dec 3, 2014 at 5:54 AM, svante karlsson <saka@csi.se> wrote:

> I've installed (for ansible scripting testing purposes) 3 VM's each
> containing kafka & zookeeer clustered together
>
> Ubuntu 14.04
> Zookeepers are 3.4.6 and kafka 2.11-0.8.2-beta
> The kafka servers have broker id's 2, 4, 6
>
> The zookeepers seems happy.
> The kafka servers start up and seems happy.
>
> I can created two test topics as in the getting started guide
> bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 1 --partitions 1 --topic test
> bin/kafka-topics.sh --create --zookeeper localhost:2181
> --replication-factor 3 --partitions 32 --topic test3
>
> I published some messages on each topic and after a while I noticed that I
> ran out of disk space and there are millions of logs in syslog similar to
> the one below.
>
> Any hints on what can cause this problem?
>
>
> /svante
>
>
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,120]
> WARN [Replica Manager on Broker 4]: Fetch request with  correlation id 21
> from client ReplicaFetcherThread-0-6 on partition [test3,3] failed due to
> Leader not local for partition [test3, 3] on broker 4
> (kafka.server.ReplicaManager)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,120]
> WARN [Replica Manager on Broker 4]: Fetch request with correlation id 21
> from client ReplicaFetcherThread-0-6 on partition [test3,4] failed due to
> Leader not local for partition [test3, 4] on broker 4
> (kafka.server.ReplicaManager)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,120]
> WARN [Replica Manager on Broker 4]: Fetch request with  correlation id 21
> from client ReplicaFetcherThread-0-6 on partition [test3,26] failed due to
> Leader not local for partition [test3,26] on broker 4
> (kafka.server.ReplicaManager)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,121]
> ERROR [KafkaApi-4] error when handling request Name: FetchRequest; Version:
> 0; CorrelationId: 21; ClientId: ReplicaFetcherThread-0-6; ReplicaId: 4;
> MaxWait: 500 ms; MinBytes: 1 bytes; RequestInfo: [test3,22] ->
> PartitionFetchInfo(0,1048576),[test3,31] ->
> PartitionFetchInfo(0,1048576),[test3,21]
> ->PartitionFetchInfo(0,1048576),[test3,13] ->
> PartitionFetchInfo(0,1048576),[test3,9] ->
> PartitionFetchInfo(0,1048576),[test3,28] ->
> PartitionFetchInfo(0,1048576),[test3,27] ->
> PartitionFetchInfo(0,1048576),[test3,15] ->
> PartitionFetchInfo(0,1048576),[test3,7] ->
> PartitionFetchInfo(0,1048576),[test3,8] ->
> PartitionFetchInfo(0,1048576),[test3,2] ->
> PartitionFetchInfo(0,1048576),[test3,19] ->
> PartitionFetchInfo(0,1048576),[test3,25] ->
> PartitionFetchInfo(0,1048576),[test3,20] ->
> PartitionFetchInfo(0,1048576),[test3,14] ->
> PartitionFetchInfo(0,1048576),[test2,0] ->
> PartitionFetchInfo(0,1048576),[test3,16] ->
> PartitionFetchInfo(0,1048576),[test3,1] ->
> PartitionFetchInfo(0,1048576),[test3,10] ->
> PartitionFetchInfo(0,1048576),[test3,3] ->
> PartitionFetchInfo(0,1048576),[test3,4] ->
> PartitionFetchInfo(0,1048576),[test3,26] -> PartitionFetchInfo(0,1048576)
> (kafka.server.KafkaApis)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker
> kafka.common.KafkaException: Should not set log end offset on partition
> [test3,22]'s local replica 4
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> kafka.cluster.Replica.logEndOffset_$eq(Replica.scala:52)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
>
> kafka.server.ReplicaManager.updateReplicaLEOAndPartitionHW(ReplicaManager.scala:565)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
>
> kafka.server.KafkaApis$$anonfun$recordFollowerLogEndOffsets$2.apply(KafkaApis.scala:348)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
>
> kafka.server.KafkaApis$$anonfun$recordFollowerLogEndOffsets$2.apply(KafkaApis.scala:346)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
>
> scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
>
> scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
>
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:778)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:221)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:428)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
>
> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:777)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> scala.collection.MapLike$MappedValues.foreach(MapLike.scala:245)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> kafka.server.KafkaApis.recordFollowerLogEndOffsets(KafkaApis.scala:346)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> kafka.server.KafkaApis.handleFetchRequest(KafkaApis.scala:311)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> kafka.server.KafkaApis.handle(KafkaApis.scala:60)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:59)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker #011at
> java.lang.Thread.run(Thread.java:745)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,124]
> ERROR [ReplicaFetcherThread-0-6], Error for partition [test3,22] to broker
> 6:class kafka.common.UnknownException(kafka.server.ReplicaFetcherThread)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,124]
> ERROR [ReplicaFetcherThread-0-6], Error for partition [test3,31] to broker
> 6:class kafka.common.UnknownExceptionkafka.server.ReplicaFetcherThread)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,124]
> ERROR [ReplicaFetcherThread-0-6], Error for partition [test3,21] to broker
> 6:class kafka.common.UnknownException(kafka.server.ReplicaFetcherThread)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,124]
> ERROR [ReplicaFetcherThread-0-6], Error for partition [test3,13] to broker
> 6:class kafka.common.UnknownException(kafka.server.ReplicaFetcherThread)
> Dec  2 07:40:17 ubuntu supervisord: kafka-broker [2014-12-02 07:40:17,124]
> ERROR [ReplicaFetcherThread-0-6], Error for partition [test3,9] to broker
> 6:class kafka.common.UnknownException(kafka.server.ReplicaFetcherThread)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message