kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lxy <lxys...@163.com>
Subject I have one partition the consumer never catch up. Why?
Date Wed, 04 Dec 2019 01:54:34 GMT
Hello, guys,
I have one topic with 40 partitions. And 40 consumers process messages from this topic, and
the logic are the same, of course.

But I find one partition the consumer never catch up, even when there is no more new message.
And It seems that this partition is fixed.
It is odd, and I do not know where to begin to debug.


Btw, Kafka version 2.3.1, and the clients use librdkafka v1.0.0.
Thank you



GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG           
 CONSUMER-ID                                       HOST            CLIENT-ID
qoe             QOE_CMA         36         782929          782930          1             
 e6347a54993b-985f7d0b-eca8-4bb9-b683-699e25ddd9f9 /172.20.1.13    e6347a54993b
qoe             QOE_CMA         23         765115          765116          1             
 9590e99ab891-8db65cc8-eeed-438e-ba53-81ed4f552f54 /172.20.1.11    9590e99ab891
qoe             QOE_CMA         2          774034          774035          1             
 0569fcc1c252-375488d6-311c-42a7-8521-7f63f9287da5 /172.20.1.15    0569fcc1c252
qoe             QOE_CMA         27         783641          783642          1             
 a20263ce011a-4e58f316-b864-4aab-837e-5719a8869d9f /172.20.1.15    a20263ce011a
qoe             QOE_CMA         31         788529          788530          1             
 aa279a3cf0c7-77778c62-c70f-4c67-a8ad-b8d54ef3fa16 /172.20.1.11    aa279a3cf0c7
qoe             QOE_CMA         6          762991          762992          1             
 226f51b1e621-2ee0f458-4969-4adb-9c8d-68649f7cac8e /172.18.0.7     226f51b1e621
qoe             QOE_CMA         9          744157          744158          1             
 2b1d20d23972-fc846b80-b001-4287-ad59-444852b74cc0 /172.20.1.14    2b1d20d23972
qoe             QOE_CMA         33         777356          777357          1             
 b410a4955c4b-8c120202-6dcc-482e-80ac-c6c4723900e4 /172.20.1.15    b410a4955c4b
qoe             QOE_CMA         34         770725          770726          1             
 c23e9a264c04-4395c441-9436-49ab-a5f1-adbce38fa8c5 /172.20.1.13    c23e9a264c04
qoe             QOE_CMA         35         774532          774533          1             
 d914e4c97d0e-43012767-4287-4aa3-b6de-1908d53b12b7 /172.18.0.6     d914e4c97d0e
qoe             QOE_CMA         7          764686          764687          1             
 243f5625f5ac-a150edbf-d6e5-4bb1-a243-1933f3a2945b /172.18.0.18    243f5625f5ac
qoe             QOE_CMA         16         780289          780290          1             
 5c51a44fb82a-458fa6dc-427f-4e30-941e-bb45594df7b9 /172.18.0.9     5c51a44fb82a
qoe             QOE_CMA         21         776035          776036          1             
 88456832e165-44e0907f-5d4d-4a32-9ac8-0047c020d596 /172.20.1.14    88456832e165
qoe             QOE_CMA         20         740141          740142          1             
 76398a57822f-ad980a19-07eb-401f-b93d-dc151a2c0f88 /172.20.1.14    76398a57822f
qoe             QOE_CMA         29         793344          793345          1             
 a414a4a64c11-6c70a97f-faa1-470a-9d8d-ec85d30d053f /172.20.1.14    a414a4a64c11
qoe             QOE_CMA         1          790367          790368          1             
 031bc74ccddd-5d666bc4-a4b4-4e9a-8d2a-b9b0725253a9 /172.20.1.14    031bc74ccddd
qoe             QOE_CMA         28         785596          785597          1             
 a3ce66702328-afe5251a-7b4b-413d-ae49-1fb04a889afc /172.20.1.15    a3ce66702328
qoe             QOE_CMA         26         759925          759926          1             
 99130ac5e7f4-88430a27-c497-405d-b0d7-141c80350da2 /172.20.1.13    99130ac5e7f4
qoe             QOE_CMA         38         759420          759421          1             
 f29b9c4299d4-7b694231-11b3-4678-a3fd-0ba316f48125 /172.20.1.14    f29b9c4299d4
qoe             QOE_CMA         14         764912          764913          1             
 53812c7e22a4-96b02e91-1efa-4e3e-9cd0-786e107b40cd /172.18.0.3     53812c7e22a4
qoe             QOE_CMA         0          769361          770110          749           
 02dafda02c49-d595f8f6-c998-4e9b-955d-b68d7ccbb45b /172.20.1.14    02dafda02c49
qoe             QOE_CMA         24         751582          751583          1             
 95927bd88f03-adf22b9e-0b32-4c07-a7ae-e4ca1dd6ed55 /172.20.1.11    95927bd88f03
qoe             QOE_CMA         15         793108          793109          1             
 59fa34c9b3b4-89ef2674-f2ee-492a-8b18-262e17206667 /172.20.1.15    59fa34c9b3b4
qoe             QOE_CMA         3          754024          754025          1             
 0ab8f622a447-c08ae4c1-ab92-4eed-bb96-8afb5b918803 /172.20.1.15    0ab8f622a447
qoe             QOE_CMA         19         781318          781319          1             
 63b22a124f56-220aa998-e2c7-4930-b049-bb7abc5a979a /172.20.1.13    63b22a124f56
qoe             QOE_CMA         17         766927          766928          1             
 5f0226a764b2-198ff4d9-cc62-4d61-9571-bc66a18ff9db /172.20.1.15    5f0226a764b2
qoe             QOE_CMA         5          778124          778125          1             
 1b669240f62b-3bca66f3-3cda-4268-a243-47078c6789e9 /172.20.1.11    1b669240f62b
qoe             QOE_CMA         12         752201          752202          1             
 42d60699dab6-8c4730b6-fc37-4885-9702-eae0ffcb859b /172.18.0.11    42d60699dab6
qoe             QOE_CMA         10         798445          798446          1             
 404f1c1dc03c-656dbea5-df95-4102-b49b-fb6b9ee959c8 /172.20.1.13    404f1c1dc03c
qoe             QOE_CMA         4          814119          814120          1             
 114c1ce5981f-16c7d5c0-5c29-44d1-a9e7-0ae3858543a6 /172.20.1.13    114c1ce5981f
qoe             QOE_CMA         11         777254          777255          1             
 41a109756c5f-3dd7d77e-e28a-4baf-aff3-0ae9ea4dd162 /172.20.1.11    41a109756c5f
qoe             QOE_CMA         13         766529          766530          1             
 506c2df73bb0-eb9ef39f-07aa-44ff-8086-f8247d2f2c51 /172.18.0.17    506c2df73bb0
qoe             QOE_CMA         22         802444          802445          1             
 8b5163afea1d-69daaad1-de8e-42ae-8130-db1db1873620 /172.20.1.11    8b5163afea1d
qoe             QOE_CMA         18         766113          766114          1             
 630bae0a586a-7220363a-88f0-47e8-b7e5-af1579de18a1 /172.20.1.13    630bae0a586a
qoe             QOE_CMA         30         779447          779448          1             
 a4b8fcadedc0-0c291f4d-7799-43c5-978b-c71bd131e410 /172.20.1.11    a4b8fcadedc0
qoe             QOE_CMA         37         760712          760713          1             
 e63cdba57f0a-a1ab8752-0122-4cda-a004-9e72d91a51d8 /172.20.1.15    e63cdba57f0a
qoe             QOE_CMA         39         757436          757437          1             
 f8164a21965f-864cbdbd-3378-4033-a6f4-44344c536d47 /172.20.1.13    f8164a21965f
qoe             QOE_CMA         25         789973          789974          1             
 95fbbb81f5b2-e46779dd-62ac-4f43-97d3-ccc9324ecebc /172.18.0.5     95fbbb81f5b2
qoe             QOE_CMA         8          785152          785153          1             
 294bcac52eb8-1a003b16-274d-4099-b465-645101913a59 /172.20.1.14    294bcac52eb8
qoe             QOE_CMA         32         782282          782283          1             
 ab61dedcceec-2bd55abc-281f-44ad-ab8e-2d0c2757017f /172.20.1.11    ab61dedcceec
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message