kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Mackles <pmack...@adobe.com>
Subject Re: JMXTrans not sending kafka 0.8 metrics to Ganglia
Date Tue, 05 Nov 2013 01:10:39 GMT
It looks like you are missing quotes in the object name. Here is a snippet
from our jmxtrans configs:

"resultAlias": "ReplicaManager",
     "obj": "\"kafka.server\":type=\"ReplicaManager\",name=\"*\"",
     "attr": [
      "Count",
      "OneMinuteRate",
      "MeanRate",
      "Value"
     ]


Unless more recent versions of kafka get rid of the quotes (we are on an
older 0.8 version still).

Paul

On 11/4/13 8:02 PM, "Priya Matpadi" <priya.matpadi@ecofactor.com> wrote:

>I am trying to send kafka metrics for display to ganglia server using
>latest download from https://github.com/adambarthelson/kafka-ganglia.
>
>Here's my kafka_metrics.json
>{
>  "servers" : [ {
>    "port" : "9999",
>    "host" : "ecokaf1",
>    "queries" : [ {
>      "outputWriters" : [ {
>        "@class" :
>"com.googlecode.jmxtrans.model.output.GangliaWriter",
>        "settings" : {
>          "groupName" : "kafka stats 2",
>          "port" : 8649,
>          "host" : "ecokaf1"
>        }
>      } ],
>      "obj" : "kafka.server:type=ReplicaManager,name=PartitionCount",
>      "resultAlias": "Kafka.ReplicaManager",
>      "attr" : [ "Value" ]
>    } ],
>    "numQueryThreads" : 2
>  } ]
>}
>
>I start jmxtrans as follows:
>sudo ./jmxtrans.sh start kafka_metrics.json
>
>
>JMXTrans starts without any issues. Here are the logs:
>
>[05 Nov 2013 00:49:48] [main] 0      INFO
>(com.googlecode.jmxtrans.JmxTransformer:134) - Starting Jmxtrans on :
>kafka_metrics.json
>[05 Nov 2013 00:49:48] [main] 386    DEBUG
>(com.googlecode.jmxtrans.JmxTransformer:354) - Loaded file:
>/usr/share/jmxtrans/kafka_metrics.json
>[05 Nov 2013 00:49:48] [main] 392    DEBUG
>(com.googlecode.jmxtrans.model.output.GangliaWriter:119) - Validated
>Ganglia metric [host: ecokaf1, port: 8649, addressingMode: UNICAST, ttl:
>5,
>v3.1: true, units: '', slope: BOTH, tmax: 60, dmax: 0, spoofedHostName:
>192.168.3.1:ecokaf1, groupName: 'kafka stats 2']
>[05 Nov 2013 00:49:48] [main] 406    DEBUG
>(com.googlecode.jmxtrans.JmxTransformer:429) - Scheduled job:
>ecokaf1:9999-1383612588479-3580845919 for server: Server [host=ecokaf1,
>port=9999, url=null, cronExpression=null, numQueryThreads=2]
>[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 414    DEBUG
>(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
>Server [host=ecokaf1, port=9999, url=null, cronExpression=null,
>numQueryThreads=2]
>[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 544    DEBUG
>(com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 1 query
>threads
>[05 Nov 2013 00:49:48] [ServerScheduler_Worker-1] 553    DEBUG
>(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
>Server [host=ecokaf1, port=9999,
>url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi,
>cronExpression=null,
>numQueryThreads=2]
>[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60401  DEBUG
>(com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
>Server [host=ecokaf1, port=9999,
>url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi,
>cronExpression=null,
>numQueryThreads=2]
>[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60404  DEBUG
>(com.googlecode.jmxtrans.util.JmxUtils:102) - ----- Creating 1 query
>threads
>[05 Nov 2013 00:50:48] [ServerScheduler_Worker-2] 60410  DEBUG
>(com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
>Server [host=ecokaf1, port=9999,
>url=service:jmx:rmi:///jndi/rmi://ecokaf1:9999/jmxrmi,
>cronExpression=null,
>numQueryThreads=2]
>
>Does any one spot an issue with the json?
>
>When I add the jvmheapmemory and jvmGC examples to my json, jmxtrans sends
>those metrics to Ganglia. Corresponding rrds files are created in ganglia
>location and I can see the following in jmxtrans.log:
>[05 Nov 2013 00:45:44] [pool-30-thread-1] 1740436 DEBUG
>(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending Ganglia
>metric heap.HeapMemoryUsage_init=263441792
>[05 Nov 2013 00:45:44] [pool-30-thread-2] 1740449 DEBUG
>(com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending Ganglia
>metric GC.PSScavenge.CollectionCount=334
>
>And I know that kafka is publishing jmx metrics because I can see them
>using jmxsh-R5 command line tool,
>
>Appreciate any pointers you can provide with regards to kafka.
>Thanks,
>Priya


Mime
View raw message