kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Achanta Vamsi Subhash <achanta.va...@flipkart.com>
Subject Re: dumping JMX data
Date Fri, 01 Apr 2016 08:11:44 GMT
Why not use tools like jmxtrans <https://github.com/jmxtrans/jmxtrans/wiki> and
send your metrics to Graphite/OpenTsdb.etc? Why do
serialization/de-serialization twice?

On Fri, Apr 1, 2016 at 9:51 AM, Gerard Klijs <gerard.klijs@dizzit.com>
wrote:

> Don't know if adding it to Kafka is a good thing. I assume you need some
> java opts settings for it to work, and with other solutions these would be
> different. It could be enabled with an option off course, then it's not in
> the way if you use something else.
> We use zabbix, this is a single tool which can be used to read in jmx data,
> store the data for a certain time, configurable for each item, and create
> triggers and graphs for those.
>
> To see and copy jmx items we use the Oracle Java mission control, it has a
> tab with info on each jmx item, which can be copied to clipboard.
>
> On Fri, Apr 1, 2016, 02:03 Sean Clemmer <sclemmer@bluejeansnet.com> wrote:
>
> > Another +1 for Jolokia. We've got a pretty cool setup here that deploys
> > Jolokia alongside Kafka, and we wrote a small Sensu plugin to grab all
> the
> > stats from Jolokia's JSON API and reformat them for Graphite.
> >
> > On Thu, Mar 31, 2016 at 4:36 PM, craig w <codecraig@gmail.com> wrote:
> >
> > > Including jolokia would be great, I've used for kafka and it worked
> well.
> > > On Mar 31, 2016 6:54 PM, "Christian Posta" <christian.posta@gmail.com>
> > > wrote:
> > >
> > > > What if we added something like this to Kafka? https://jolokia.org
> > > > I've added a JIRA to do that, just haven't gotten to it yet. Will
> soon
> > > > though, especially if it'd be useful for others.
> > > >
> > > > https://issues.apache.org/jira/browse/KAFKA-3377
> > > >
> > > > On Thu, Mar 31, 2016 at 2:55 PM, David Sidlo <DSidlo@ancestry.com>
> > > wrote:
> > > >
> > > > > The Kafka JmxTool works fine although it is not user friendly, in
> > that
> > > > you
> > > > > cannot perform a query of the Kafka Server mbeans to determine
> > content
> > > > and
> > > > > to determine the path-string that you need to place into the
> > > -object-name
> > > > > option.
> > > > >
> > > > > Here's how I solved the problem...
> > > > >
> > > > > First, make sure that Kafka is running with jms options enabled...
> > > > >
> > > > > -          The following opens up the jxm port with no
> authentication
> > > > (for
> > > > > testing)...
> > > > > -Dcom.sun.management.jmxremote
> > -Dcom.sun.management.jmxremote.port=9999
> > > > > -Dcom.sun.management.jmxremote.ssl=false
> > > > > -Dcom.sun.management.jmxremote.authenticate=false
> > > > >
> > > > > Second, get jstatd running on the same server so that you can use
> > > > VisualVM
> > > > > to look into what is going on inside.
> > > > >
> > > > > Then, use VisualVM along with its jmx-plugin to view the mbeans and
> > > > > contents.
> > > > >
> > > > > When using VisualVM, you will first connect to jstatd, then you
> have
> > to
> > > > > right click on the host to request a JMX connection, where you will
> > > get a
> > > > > dialog, where you have to add the jmx port number to the host name
> > > (after
> > > > > the colon).
> > > > >
> > > > > When you view the mbeans, you will see the path to the given
> > attribute
> > > if
> > > > > you hover over it with the mouse pointer, but only for a moment...
> > The
> > > > > string can be quite long and... Unfortunately, you don't have the
> > > option
> > > > to
> > > > > capture that string into the cut-buffer via the interface. I used
a
> > > > screen
> > > > > capture utility to capture the string and typed it into the
> terminal.
> > > > >
> > > > > So here is what my first working query looked like...
> > > > >
> > > > > /opt/kafka/kafka/bin/kafka-run-class.sh kafka.tools.JmxTool
> > > --object-name
> > > > >
> > > >
> > >
> >
> "kafka.consumer:type=FetchRequestAndResponseMetrics,name=FetchRequestRateAndTimeMs,clientId=ReplicaFetcherThread-0-5,brokerHost=
> > > > > hostname05.cluster.com,brokerPort=9092" --jmx-url
> > > > > service:jmx:rmi:///jndi/rmi://`hostname`:9999/jmxrmi
> > > > >
> > > > > That command will output all of the attributes for the given
> > > object-name,
> > > > > and it's a long object-name.
> > > > > With some experimentation, I found that you can use wild-cards on
> > > > portions
> > > > > of the object-name that were clearly dynamic, such as the host-name
> > or
> > > > the
> > > > > thread-name. So you can get the attribute for all similar objects
> by
> > > > using
> > > > > the following query...
> > > > >
> > > > > /opt/kafka/kafka/bin/kafka-run-class.sh kafka.tools.JmxTool
> > > --object-name
> > > > >
> > > >
> > >
> >
> "kafka.consumer:type=FetchRequestAndResponseMetrics,name=FetchRequestRateAndTimeMs,clientId=ReplicaFetcherThread*,brokerHost=hostname*.
> > > > > cluster.com,brokerPort=*" --jmx-url
> > > > > service:jmx:rmi:///jndi/rmi://`hostname`:9999/jmxrmi
> > > > >
> > > > > There may be simpler tools to use, but I really do like the GUI
> > > goodness
> > > > > in VisualVM, (and it is free).
> > > > >
> > > > > I hope that helps.
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > *Christian Posta*
> > > > twitter: @christianposta
> > > > http://www.christianposta.com/blog
> > > > http://fabric8.io
> > > >
> > >
> >
>



-- 
Regards
Vamsi Subhash

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message