kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brett Rann <br...@zendesk.com.INVALID>
Subject Re: Kafka 0.10.0.2 reset offset in the new-consumer mode
Date Thu, 23 Nov 2017 12:02:48 GMT
Ah apologies.

Found the KIP where the one I suggested was added:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-122%3A+Add+Reset+Consumer+Group+Offsets+tooling

And from the discussion thread link, a couple in to the thread:

Currently the only way for an admin to successfully override offsets is if
either:
1. they send an OffsetCommitRequest with generationId -1 and if the group
is in the Empty state
2. they send an OffsetCommitRequest impersonating a member of the group
with accurate generationId and memberId

You can actually get away with overriding while members of the group are
live with method 2 by using group information from DescribeGroupsRequest.



On Thu, Nov 23, 2017 at 9:20 PM, Ali Nazemian <alinazemian@gmail.com> wrote:

> Unfortunately, it doesn't have that option in this version of Kafka!
>
> On Thu, Nov 23, 2017 at 9:02 PM, Brett Rann <brann@zendesk.com.invalid>
> wrote:
>
> > I don't know about kafka-storm spout, but you could try using
> > the kafka-consumer-groups.sh cli to reset the offset. It has
> > a --reset-offsets option.
> >
> > On Thu, Nov 23, 2017 at 7:02 PM, Ali Nazemian <alinazemian@gmail.com>
> > wrote:
> >
> > > Hi All,
> > >
> > > I am using Kafka 0.10.0.2 and I am not able to upgrade my Kafka
> version.
> > I
> > > have a situation that after removing Kafka topic, I am getting the
> > > following error in Kafka-Storm Spout client because the offset hasn't
> > been
> > > reset properly. I was wondering how I can reset the offset in the
> > > new-consumer mode with this version of Kafka. Since I am using
> > > new-consumer, Kafka does not maintain offset info in Zookeeper.
> > >
> > > 2017-11-23 07:55:52.870 o.a.s.k.s.i.OffsetManager [WARN]
> > > topic-partition [indexing-1] has unexpected offset [7259]. Current
> > > committed Offset [8855387]
> > >
> > >
> > > Regards,
> > > Ali
> > >
> >
>
>
>
> --
> A.Nazemian
>



-- 

Brett Rann

Senior DevOps Engineer


Zendesk International Ltd

395 Collins Street, Melbourne VIC 3000 Australia

Mobile: +61 (0) 418 826 017

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message