kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vincent Maurin <vincent.maurin...@gmail.com>
Subject Re: __consumer_offsets topic, some partitions with the last offset equals to zero
Date Tue, 09 Apr 2019 12:27:48 GMT
Hello Jonathan,

In the __consumer_offsets, the messages have a key with (group, topic,
partition) and the last consumed offset as value.
If you have less than 50 group/topic/partition combination in your cluster,
it could make sense to have __consumer_offsets partitions without any
message

Best regards

On Tue, Apr 9, 2019 at 12:43 PM Jonathan Santilli <
jonathansantilli@gmail.com> wrote:

> Hello, hope you are doing well,
>
> We have noticed that the last offset for some of the partitions for
> topic *__consumer_offsets
> is zero*. For instance, the following is a testing cluster running version
> 2.2, this cluster has been running for about 15 months already:
>
> *./bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list
> localhost:9092 -time -1 --topic __consumer_offsets*
>
> *__consumer_offsets:0:0*
> *__consumer_offsets:1:0*
> *__consumer_offsets:2:0*
> *__consumer_offsets:3:0*
> *__consumer_offsets:4:0*
> *__consumer_offsets:5:105442*
> *__consumer_offsets:6:1360081*
> *__consumer_offsets:7:0*
> *__consumer_offsets:8:214*
> *__consumer_offsets:9:0*
> *__consumer_offsets:10:1235*
> *__consumer_offsets:11:0*
> *__consumer_offsets:12:3*
> *__consumer_offsets:13:336*
> *__consumer_offsets:14:53858848*
> *__consumer_offsets:15:0*
> *__consumer_offsets:16:0*
> *__consumer_offsets:17:482*
> *__consumer_offsets:18:4660003*
> *__consumer_offsets:19:236*
> *__consumer_offsets:20:227*
> *__consumer_offsets:21:0*
> *__consumer_offsets:22:273*
> *__consumer_offsets:23:3*
> *__consumer_offsets:24:0*
> *__consumer_offsets:25:148363133*
> *__consumer_offsets:26:0*
> *__consumer_offsets:27:103*
> *__consumer_offsets:28:7*
> *__consumer_offsets:29:0*
> *__consumer_offsets:30:3*
> *__consumer_offsets:31:0*
> *__consumer_offsets:32:0*
> *__consumer_offsets:33:134*
> *__consumer_offsets:34:95*
> *__consumer_offsets:35:0*
> *__consumer_offsets:36:0*
> *__consumer_offsets:37:550*
> *__consumer_offsets:38:0*
> *__consumer_offsets:39:0*
> *__consumer_offsets:40:1*
> *__consumer_offsets:41:85*
> *__consumer_offsets:42:0*
> *__consumer_offsets:43:0*
> *__consumer_offsets:44:6*
> *__consumer_offsets:45:0*
> *__consumer_offsets:46:0*
> *__consumer_offsets:47:65069112*
> *__consumer_offsets:48:930655*
> *__consumer_offsets:49:108*
>
>
> As you can see, is common to get partitions with the last offset *as zero*,
> is that expected or normal? am wondering also because, the default
> partitions number for this *topic is 50*, so, I will assume that by design
> all of them should be used.
>
> What am missing?
>
>
> Cheers!
> --
> Santilli Jonathan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message