kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kumar Jayanti <kumar.jaya...@gmail.com>
Subject Re: Finding consumer Lag
Date Mon, 08 Jun 2020 16:47:42 GMT
Hi,
Did you checkout the following command
kafka-consumer-groups.sh --zookeeper zkhost:2181/kafka-cluster --describe
--group groupname

This will return the LAG information and it is defined as :
“the difference between the consumer Current-Offset and the broker
Log-End-Offset for this topic partition”

On Mon, Jun 8, 2020 at 6:48 PM Indu V <induv@suntecgroup.com> wrote:

> Hi Team,
>
> I am facing an issue in finding the lag (unconsumed record count) from
> Kafka against a topic. I have another process that has to be run after all
> records are consumed.
>
> Each topic has given a single partition with the replication factor as 1.
>
> Lag is computed by taking the difference of published offset and consumed
> offset.
>
> In a transactional scenario, the published offset which is the partition
> end offset includes control messages. In my understanding, this is
> internally added by Kafka to handle the transaction commit and rollback.
>
> I got count mismatches in a transaction rollback scenario. For a single
> unconsumed record, it is showing lag as 3.
>
> So is there any way I can exclude the control messages from the lag.
>
> Regards,
> Indu V
>
> This electronic mail (including any attachment thereto) may be
> confidential and privileged and is intended only for the individual or
> entity named above. Any unauthorized use, printing, copying, disclosure or
> dissemination of this communication may be subject to legal restriction or
> sanction. Accordingly, if you are not the intended recipient, please notify
> the sender by replying to this email immediately and delete this email (and
> any attachment thereto) from your computer system...Thank You.
>


-- 
~~~~~~~~~~~~~~~~~~~~~~
https://kumarjayanti.github.io

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message