kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Palino <tpal...@linkedin.com>
Subject Re: What do the numbers in the ConsumerOffsetChecker mean
Date Fri, 04 Apr 2014 13:31:21 GMT
The numbers are offsets, which means that the lag is computed as ³number
of messages², not bytes. I know the header ³logSize² can be a little
confusing with regards to that.

-Todd


On 4/4/14, 12:32 AM, "S L" <slouie.at.work@gmail.com> wrote:

>Hi,
>
>Just wondering what the units for the results from the
>ConsumerOffsetChecker are.
>
>For example, this was the output when we ran the OffsetChecker:
>Group           Topic                          Pid Offset          logSize
>        Lag             Owner
>xml1      xmlnew                      0   28317611        32809085
> 4491474         none
>
>Is the lag in bytes or is it an offset pointer of where it's at in the
>topic, or am I totally off and it means something else?  Just asking
>because one time our consumer was stopped for about 30 min but the lag
>value / logSize didn't grow as much as we thought it should in a 30 min
>period.  We know we should get 100+MB of data a minute but the Lag value
>was something like 60000 when we checked after 30 min.
>
>We looked at this site
>https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemst
>ohavestopped%2Cwhy%3F
>
>which makes it look like it represents bytes, yet looking at the
>kafka.apache.org (
>http://kafka.apache.org/documentation.html#basic_ops_consumer_lag) site
>makes it sound like it represents a position in the topic.
>
>Thanks in advance for your help.


Mime
View raw message