kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kim Chew <kchew...@gmail.com>
Subject Topic partitions logSize are way behind the offsets.
Date Tue, 24 Apr 2018 00:19:12 GMT
The following is the output from "kafka-consumer-offset-checker.sh",

Group           Topic                          Pid Offset
logSize         Lag             Owner
secorV2_backup  auth-data                      0   13433
2               -13431          secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup  auth-data                      1   17466
17              -17449          secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup  auth-data                      2   15284
11              -15273          secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup  auth-data                      3   10174
14              -10160          secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup  auth-data                      4   24228
44              -24184          secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup  auth-data                      5   5239
1               -5238           secorV2_backup_ip-172-30-13-99_3448_243-0
secorV2_backup  auth-data                      6   653137
10921           -642216         secorV2_backup_ip-172-30-13-99_3448_243-0
...............

It happened after the kafka cluster has restarted.

I am trying to understand why there are such a huge "Lag" values for some
of the partitions and the logSize are so small. What are some of the
reasons behind it? And what would be the good ways to fix it? Because just
waiting kafka to "catch up" with the offsets, where in this case, may be
weeks.

TIA.

Kim

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message