kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Consumer Group Lag Reporting
Date Tue, 07 Apr 2015 04:05:22 GMT

We track our Consumer Lag and all other Kafka metrics with SPM
<http://sematext.com/spm> whose agent collects this info from Kafka
Consumers.  Here's a chart of that:
https://apps.sematext.com/spm-reports/s/mcBHvsXwgj -- this public chart
doesn't show all filtering option, but you can filter lag by host, client
id, topic, and partition.

Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/

On Mon, Apr 6, 2015 at 6:14 PM, Kyle Banker <kylebanker@gmail.com> wrote:

> What is the best practice for reporting the lag on individual consumer
> groups (e.g., to Graphite)?
> A recent form post (http://search-hadoop.com/m/4TaT4x9qWm1) seems to
> indicate that parsing the output of the consumer offset checker tool and
> reporting that independently is what folks do. Is there a better way?
> Are there any plans for the Kafka brokers themselves to do this reporting
> (e.g., the master for each partition reports lag for all of its consumer
> groups)?
> Many thanks in advance.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message