kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Montgomery <davidmontgom...@gmail.com>
Subject Re: Re: How to push metrics to graphite - jmxtrans does not work
Date Wed, 03 Dec 2014 06:35:03 GMT
Hi,

Thanks for the help.  I found the issue.....I was appending to the bottom
when I should have placed the below line at the top of the file.


echo 'KAFKA_JMX_OPTS="-Dcom.sun.
management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false"' | tee -a
/var/kafka/bin/kafka-run-class.sh


On Wed, Dec 3, 2014 at 1:27 PM, Jason Rosenberg <jbr@squareup.com> wrote:

> fwiw, we wrap the kafka server in our java service container framework.
> This allows us to use the default GraphiteReporter class that is part of
> the yammer metrics library (which is used by kafka directly).  So it works
> seemlessly.  (We've since changed our use of GraphiteReporter to instead
> send all our metrics via kafka :))....
>
> Jason
>
> On Tue, Dec 2, 2014 at 11:00 PM, Otis Gospodnetic <
> otis.gospodnetic@gmail.com> wrote:
>
> > Hi,
> >
> > You can make use of this documentation aimed at JMX and monitoring:
> >
> https://sematext.atlassian.net/wiki/display/PUBSPM/SPM+Monitor+-+Standalone
> >
> > There is a section about Kafka and the information is not SPM-specific.
> >
> > Otis
> > --
> > Monitoring * Alerting * Anomaly Detection * Centralized Log Management
> > Solr & Elasticsearch Support * http://sematext.com/
> >
> >
> > On Tue, Dec 2, 2014 at 9:34 PM, YuanJia Li <yuanjia8947@163.com> wrote:
> >
> > > Hi David´╝î
> > >     Just edit "kafka-server-start.sh", and add "export
> JMX_PORT=9999",it
> > > will work.
> > >
> > >
> > >
> > >
> > > Yuanjia
> > >
> > > From: David Montgomery
> > > Date: 2014-12-03 04:47
> > > To:users
> > > Subject: Re: How to push metrics to graphite - jmxtrans does not work
> > > Hi,
> > >
> > > I am seeing this in the logs and wondering what "jmx_port":-1 means?
> > >
> > > INFO conflict in /brokers/ids/29136 data: { "host":"104.111.111.111.",
> > > "jmx_port":-1, "port":9092, "timestamp":"1417552817875", "version":1 }
> > > stored data: { "host":"104.111.111", "jmx_port":-1, "port":9092,
> > > "timestamp":"1417552738253", "version":1
> > >
> > > despite having these added
> > >
> > > echo 'KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote.port=9999
> > > -Dcom.sun.management.jmxremote=true
> > > -Dcom.sun.management.jmxremote.authenticate=false
> > > -Dcom.sun.management.jmxremote.ssl=false"' | tee -a
> > > /var/kafka/bin/kafka-run-class.sh
> > > echo 'export JMX_PORT=${JMX_PORT:-9999}' | tee -a
> > > /var/kafka/bin/kafka-server-start.sh
> > >
> > > Thanks
> > >
> > > On Tue, Dec 2, 2014 at 9:58 PM, Andrew Otto <aotto@wikimedia.org>
> wrote:
> > >
> > > > Maybe also set:
> > > >
> > > >  -Dcom.sun.management.jmxremote.port=9999
> > > >
> > > > ?
> > > >
> > > >
> > > > > On Dec 2, 2014, at 02:59, David Montgomery <
> > davidmontgomery@gmail.com>
> > > > wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I am having a very difficult time trying to report kafka 8 metrics
> to
> > > > > Graphite.  Nothing is listening on 9999 and and no data in
> graphite.
> > > If
> > > > > this method of graphite reporting is know to not work is there an
> > > > > alternative to jmxtrans to get data to graphite?
> > > > >
> > > > > I am using the deb file to install jmxtrans on ubuntu 12.04
> > > > >
> > > > > And I use the below to modify kafka scripts
> > > > >
> > > > > echo 'KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true
> > > > > -Dcom.sun.management.jmxremote.authenticate=false
> > > > > -Dcom.sun.management.jmxremote.ssl=false"' | tee -a
> > > > > /var/kafka/bin/kafka-run-class.sh
> > > > > echo 'export JMX_PORT=${JMX_PORT:-9999}' | tee -a
> > > > > /var/kafka/bin/kafka-server-start.sh
> > > > >
> > > > > {
> > > > >  "servers" : [ {
> > > > >    "host" : "127.0.0.1",
> > > > >    "port" : "9999",
> > > > >    "alias" : "<%=node.name%>",
> > > > >    "queries" : [
> > > > > {
> > > > >         "obj" : "kafka:type=kafka.SocketServerStats",
> > > > >          "resultAlias": "kafka.socketServerStats",
> > > > >          "attr" : [ "AvgFetchRequestMs", "AvgProduceRequestMs",
> > > > > "BytesReadPerSecond", "BytesWrittenPerSecond",
> > > "FetchRequestsPerSecond",
> > > > > "MaxFetchRequestMs", "MaxProduceRequestMs" , "NumFetchRequests" ,
> > > > > "NumProduceRequests" , "ProduceRequestsPerSecond",
> "TotalBytesRead",
> > > > > "TotalBytesWritten", "TotalFetchRequestMs", "TotalProduceRequestMs"
> > ],
> > > > >     "outputWriters" : [ {
> > > > >          "@class" :
> > > > "com.googlecode.jmxtrans.model.output.GraphiteWriter",
> > > > >          "settings" : {
> > > > >            "host" : "<%=@monitor_host%>",
> > > > >            "port" : "2003"
> > > > >          }
> > > > >        } ]
> > > > >      }
> > > > >    ],
> > > > >    "numQueryThreads": "2"
> > > > >  } ]
> > > > > }
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message