kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shargan <ka...@shargan.me>
Subject Re: Can't reset consumer group offsets in Kafka 0.11.0.1
Date Thu, 21 Sep 2017 18:37:32 GMT
Testing kafka-consumer-groups.sh in my dev environment, I'm unable to reset
offsets even when CURRENT-OFFSET is inbounds. Again, it returns as if the
change took effect but describe still shows the original value and client
behavior bears this out.

-shargan

On Wed, Sep 20, 2017 at 13:40 shargan <kafka@shargan.me> wrote:

> Hi there,
>
> The Kafka servers in question were 0.11.0.0 this morning. I upgraded to
> 0.11.0.1 after discovering this issue but I don’t see any change. Scala
> 2.11.
>
> A client using the new-style consumer groups is not consuming any new
> messages because its current committed offset is out of range. I shut down
> the clients to ensure the consumer group was inactive and checked the
> offset state:
>
> $ kafka-consumer-groups.sh --new-consumer --bootstrap-server
> kafka.service:9092 --group router —describe | egrep 'TOPIC|app-router-logs’
> Note: This will only show information about consumers that use the Java
> consumer API (non-ZooKeeper-based consumers).
> Consumer group 'router' has no active members.
> TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET
>  LAG        CONSUMER-ID                                       HOST
>                   CLIENT-ID
> app-router-logs                0          728660106       2324523934
>  1595863828 -                                                 -
>                  -
> app-router-logs                1          728342585       2324526718
>  1596184133 -                                                 -
>                  -
> app-router-logs                2          728698812       2324430267
>  1595731455 -                                                 -
>                  -
>
>
>
>
> Resetting the offsets acts like it worked, but doesn’t:
>
> $ kafka-consumer-groups.sh --new-consumer --bootstrap-server
> kafka.service:9092 --group router --reset-offsets --topic app-router-logs
> --to-latest
> Note: This will only show information about consumers that use the Java
> consumer API (non-ZooKeeper-based consumers).
>
> TOPIC                          PARTITION  NEW-OFFSET
> app-router-logs                2          2325666004
> app-router-logs                1          2325765832
> app-router-logs                0          2325763132
>
>
>
> kafka-consumer-groups.sh --new-consumer --bootstrap-server
> kafka.service:9092 --group router --describe | egrep 'TOPIC|app-router-logs'
> Note: This will only show information about consumers that use the Java
> consumer API (non-ZooKeeper-based consumers).
> Consumer group 'router' has no active members.
> TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET
>  LAG        CONSUMER-ID                                       HOST
>                   CLIENT-ID
> app-router-logs                0          728660106       2326329513
>  1597669407 -                                                 -
>                  -
> app-router-logs                1          728342585       2326334386
>  1597991801 -                                                 -
>                  -
> app-router-logs                2          728698812       2326237143
>  1597538331 -                                                 -
>                  -
>
>
>
> I’ve also tried —to-earliest, —to-current, and individually setting each
> partition via —to-offset and there’s no difference. What’s going on here?
>
>
> -shargan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message