kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Stopford <...@confluent.io>
Subject Re: Halting because log truncation is not allowed for topic __consumer_offsets
Date Mon, 19 Dec 2016 11:46:54 GMT
Hi Jun

This should only be possible in situations where there is a crash or
something happens to the underlying disks (assuming clean leader election).
I've not come across others. The assumption, as I understand it, is that
the underlying issue stems from KAFKA-1211
<https://issues.apache.org/jira/browse/KAFKA-1211> which is being addressed
in KIP-101
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-101+-+Alter+Replication+Protocol+to+use+Leader+Generation+rather+than+High+Watermark+for+Truncation>.
If you can reproduce in a more generally scenario we would be very
interested.

All the best
B


On Mon, Dec 19, 2016 at 12:35 AM Jun MA <mj.saber1990@gmail.com> wrote:

> Would be grateful to hear opinions from experts out there. Thanks in
> advance
>
> > On Dec 17, 2016, at 11:06 AM, Jun MA <mj.saber1990@gmail.com> wrote:
> >
> > Hi,
> >
> > We saw the following FATAL error in 2 of our brokers (3 in total, the
> active controller doesn’t have this) and they crashed in the same time.
> >
> > [2016-12-16 16:12:47,085] FATAL [ReplicaFetcherThread-0-3], Halting
> because log truncation is not allowed for topic __consumer_offsets, Current
> leader 3's latest offset 5910081 is less than replica 1's latest offset
> 5910082 (kafka.server.ReplicaFetcherThread)
> >
> > Our solution is set topic __consumer_offsets
> unclean.leader.election.enable=true temporarily, and restart brokers. In
> this way we potentially lose offsets of some topics. Is there any better
> solutions?
> >
> > I saw related tickets https://issues.apache.org/jira/browse/KAFKA-3861 <
> https://issues.apache.org/jira/browse/KAFKA-3861>,
> https://issues.apache.org/jira/browse/KAFKA-3410 <
> https://issues.apache.org/jira/browse/KAFKA-3410> and understand why
> brokers crashed. But we didn’t see any scenarios mentioned in above
> tickets. Is there any other reason why this happened? There’s no broker
> restart involved in our case. How can we prevent it from happening?
> >
> > We’re using 0.9.0 with unclean.leader.election.enable=false and
> min.insync.replicas=2.
> >
> > Thanks,
> > Jun
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message