kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun MA <mj.saber1...@gmail.com>
Subject Halting because log truncation is not allowed for topic __consumer_offsets
Date Mon, 19 Dec 2016 00:35:42 GMT
Would be grateful to hear opinions from experts out there. Thanks in advance

> On Dec 17, 2016, at 11:06 AM, Jun MA <mj.saber1990@gmail.com> wrote:
> 
> Hi,
> 
> We saw the following FATAL error in 2 of our brokers (3 in total, the active controller
doesn’t have this) and they crashed in the same time. 
> 
> [2016-12-16 16:12:47,085] FATAL [ReplicaFetcherThread-0-3], Halting because log truncation
is not allowed for topic __consumer_offsets, Current leader 3's latest offset 5910081 is less
than replica 1's latest offset 5910082 (kafka.server.ReplicaFetcherThread)
> 
> Our solution is set topic __consumer_offsets unclean.leader.election.enable=true temporarily,
and restart brokers. In this way we potentially lose offsets of some topics. Is there any
better solutions?
> 
> I saw related tickets https://issues.apache.org/jira/browse/KAFKA-3861 <https://issues.apache.org/jira/browse/KAFKA-3861>,
https://issues.apache.org/jira/browse/KAFKA-3410 <https://issues.apache.org/jira/browse/KAFKA-3410>
and understand why brokers crashed. But we didn’t see any scenarios mentioned in above tickets.
Is there any other reason why this happened? There’s no broker restart involved in our case.
How can we prevent it from happening?
> 
> We’re using 0.9.0 with unclean.leader.election.enable=false and min.insync.replicas=2.
> 
> Thanks,
> Jun


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message