kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: Opening up Kafka JMX port for Kafka Consumer in Kafka Streams app
Date Wed, 03 Aug 2016 22:03:34 GMT
Hello Phillip,

You are right that a Kafka Streams instance has one producer and consumer
client behind the scene, and hence you can just monitor it like you are
monitoring a normal producer / consumer client.

More specifically, you can find the producer metric names as in:


And Kafka Streams has a few more metrics for its own in addition to the
consumer and producer clients, for which we are yet to add the docs for.


On Tue, Aug 2, 2016 at 11:49 AM, David Garcia <davidg@spiceworks.com> wrote:

> Have you looked at kafka manager: https://github.com/yahoo/kafka-manager
> It provides consumer level metrics.
> -David
> On 8/2/16, 12:36 PM, "Phillip Mann" <pmann@trulia.com> wrote:
>     Hello all,
>     This is a bit of a convoluted title but we are trying to set up
> monitoring on our Kafka Cluster and Kafka Streams app.  I currently have
> JMX port open on our Kafka cluster across our brokers.  I am able to use a
> Java JMX client to get certain metrics that are helpful to us.  However,
> the data we want to monitor the most is consumer level JMX metrics.  This
> is made complicated because there is no documentation for what we are
> trying to do.  Essentially we need to expose the JMX port for the Kafka
> Streams Consumer.  Each Kafka Streams app contains a producer and consumer
> (group?).  We want to get the offset lag metrics for the Kafka Streams
> consumer.  Is this possible?  If so, how do we do it?  Thanks for the help!!
>     Phillip

-- Guozhang

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message