kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From liu brent <liubaoc...@gmail.com>
Subject Re: Use Kafka to collect JVM and JMX MBean's statistics.
Date Mon, 16 Apr 2012 03:16:15 GMT
Thanks guys, I think tcollector meets our needs now. :)

On Tue, Apr 10, 2012 at 12:43 AM, Dave Barr <dave.barr@gmail.com> wrote:

> We have an agent (tcollector) that collects various metrics and sends
> them to OpenTSDB.  We then have a simple python script that lets you
> query OpenTSDB for a given metric and alert on a threshold.  We
> currently gather about 1300 metrics and about 10000 datapoints/sec.
>
> https://github.com/stumbleupon/tcollector
> http://opentsdb.net/nagios.html
>
> We're just starting to use Kafka here.  I'm not aware yet of a Kafka
> collector, but I'm sure one will be coming soon.  They're really
> simple to write.  You can look at either the elasticsearch collector
> or the HBase collector as an example of different styles.  The HBase
> one forks a jmx process to read metrics from the running JVM, the
> elasticsearch one queries the running process for metrics via an
> ElasticSearch endpoint.
>
> --Dave
> On Mon, Apr 9, 2012 at 12:30 AM, liu brent <liubaochun@gmail.com> wrote:
> > Hi,
> >  Currently, there are 20+ kinds of web service running on hundreds of
> > machines in 4 data centers, which are written in java. We want to collect
> > and store the statics of JVM, MBeans of JMX, cpu usage etc of the service
> > and machines, and then send the waring to Nagios.  Does anyone has the
> > experience in using Kafka to do so?
> >  The direct solution seems like that we install a client on every
> machine,
> > which works as JMX client, collect the information and work as kafka
> > producer. Anyone has better ideas or suggestions.
> >
> > Thanks,
> > Liu
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message