kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shargan <ka...@shargan.me>
Subject Can't reset consumer group offsets in Kafka 0.11.0.1
Date Wed, 20 Sep 2017 20:50:41 GMT
Hi there,

The Kafka servers in question were 0.11.0.0 this morning. I upgraded to
0.11.0.1 after discovering this issue but I don’t see any change. Scala
2.11.

A client using the new-style consumer groups is not consuming any new
messages because its current committed offset is out of range. I shut down
the clients to ensure the consumer group was inactive and checked the
offset state:

$ kafka-consumer-groups.sh --new-consumer --bootstrap-server
kafka.service:9092 --group router —describe | egrep 'TOPIC|app-router-logs’
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
Consumer group 'router' has no active members.
TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET
 LAG        CONSUMER-ID                                       HOST
                  CLIENT-ID
app-router-logs                0          728660106       2324523934
 1595863828 -                                                 -
                 -
app-router-logs                1          728342585       2324526718
 1596184133 -                                                 -
                 -
app-router-logs                2          728698812       2324430267
 1595731455 -                                                 -
                 -




Resetting the offsets acts like it worked, but doesn’t:

$ kafka-consumer-groups.sh --new-consumer --bootstrap-server
kafka.service:9092 --group router --reset-offsets --topic app-router-logs
--to-latest
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).

TOPIC                          PARTITION  NEW-OFFSET
app-router-logs                2          2325666004
app-router-logs                1          2325765832
app-router-logs                0          2325763132



kafka-consumer-groups.sh --new-consumer --bootstrap-server
kafka.service:9092 --group router --describe | egrep 'TOPIC|app-router-logs'
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
Consumer group 'router' has no active members.
TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET
 LAG        CONSUMER-ID                                       HOST
                  CLIENT-ID
app-router-logs                0          728660106       2326329513
 1597669407 -                                                 -
                 -
app-router-logs                1          728342585       2326334386
 1597991801 -                                                 -
                 -
app-router-logs                2          728698812       2326237143
 1597538331 -                                                 -
                 -



I’ve also tried —to-earliest, —to-current, and individually setting each
partition via —to-offset and there’s no difference. What’s going on here?

-shargan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message