kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benson Chen <Benson.C...@sas.com>
Subject MBean for records-lag-max metric always 1.0 or -Infinity
Date Tue, 23 May 2017 19:11:00 GMT
Hello,

We're researching autoscaling our microservices consuming from a Kafka Topic based on the
records-lag-max metric.  For the following MBean object:

 MBeans TAB: kafka.consumer-> consumer-fetcher-manager-metric -> consumer-1 -> Attributes
-> records-lag-max

In JConsole, we noticed that we consistently get either 1.0 or -Infinity.  We also implemented
in our Java code to invoke the metrics() method of KafkaConsumer.java to fetch the metrics
after a poll of records from the Topic.
Map<MetricName,? extends Metric> metrics()    //Get the metrics kept by the consumer
Printing out this record also shows 1.0 or -Infinity.
In our test we have a Kafka Producer sending in about 50 messages/s while a Kafka Consumer
is consuming about 20 messages/s.  We know that the Kafka Consumer is definitely lagging behind
because it continues to consume messages well after shutting down the Kafka Producer.
Is this a known bug to get the correct value for records-lag-max or do we need to tune the
configuration settings somehow?
We're running 0.10.1.1.
Thanks for your help.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message