kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <joe.st...@stealth.ly>
Subject Re: metric-kafka problems
Date Sat, 10 Jan 2015 01:53:38 GMT
Hi, https://github.com/stealthly/metrics-kafka is a project to be used as
an example of how to use Kafka as a central point to send all of your
metrics for your entire infrastructure. The consumers integrate so as to
abstract the load and coupling of services so systems can just send their
stats to Kafka and then you can do whatever you want with them from there
(often multiple things). We also build a Yammer Metrics Reporter (which is
what Kafka uses to send its Metrics) for Kafka itself so brokers can send
their stats into a Kafka topic and used downstream (typically another
cluster).  The issue you reported was caused by changes by github and I
just pushed fixes for them so things are working again.

If you are not looking for that type of solution and want to just see and
chart broker metrics then I would suggest taking a look at
https://github.com/airbnb/kafka-statsd-metrics2 and you point it to
https://github.com/kamon-io/docker-grafana-graphite. I find this a very
quick out the box way to see what is going on with a broker when no stats
reporter is already in place. If you want a Kafka metrics reporter for just
graphite check out https://github.com/damienclaveau/kafka-graphite for just
ganglie https://github.com/criteo/kafka-ganglia for just Riemann
https://github.com/TheLadders/KafkaRiemannMetricsReporter and/or you also
can use a service like SPM
https://apps.sematext.com/spm-reports/mainPage.do?selectedApplication=4293
or DataDog https://www.datadoghq.com/

Hope this help, thanks!

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/

On Fri, Jan 9, 2015 at 7:51 PM, Sa Li <salicn@gmail.com> wrote:

> Hello, all
>
> I like to use the tool metrics-kafka which seems to be attractive to report
> kafka metric and use graphite to graph metrics, however I am having trouble
> to make it work.
>
> In https://github.com/stealthly/metrics-kafka, it says:
>
> In the main metrics-kafka folder
>
> 1) sudo ./bootstrap.sh 2) ./gradlew test 3) sudo ./shutdown.sh
> When I run ./bootstrap, see this is what I got
> root@DO-mq-dev:/home/stuser/jmx/metrics-kafka# ././bootstrap.sh
> /dev/stdin: line 1: syntax error near unexpected token `newline'
> /dev/stdin: line 1: `<!DOCTYPE html>'
> /dev/stdin: line 1: syntax error near unexpected token `newline'
> /dev/stdin: line 1: `<!DOCTYPE html>'
> e348a98a5afb8b89b94fce51b125e8a2045d9834268ec64c3e38cb7b165ef642
> 2015/01/09 16:49:21 Error response from daemon: Could not find entity for
> broker1
>
> And this is how I vagrant up:
> root@DO-mq-dev:/home/stuser/jmx/metrics-kafka# vagrant up
> /usr/share/vagrant/plugins/provisioners/docker/plugin.rb:13:in
> `require_relative':
> /usr/share/vagrant/plugins/provisioners/docker/config.rb:23: syntax error,
> unexpected tPOW (SyntaxError)
>       def run(name, **options)
>                       ^
> /usr/share/vagrant/plugins/provisioners/docker/config.rb:43: syntax error,
> unexpected keyword_end, expecting $end
>         from /usr/share/vagrant/plugins/provisioners/docker/plugin.rb:13:in
> `block in <class:Plugin>'
>         from /usr/lib/ruby/vendor_ruby/vagrant/registry.rb:27:in `call'
>         from /usr/lib/ruby/vendor_ruby/vagrant/registry.rb:27:in `get'
>         from
> /usr/share/vagrant/plugins/kernel_v2/config/vm_provisioner.rb:34:in
> `initialize'
>         from /usr/share/vagrant/plugins/kernel_v2/config/vm.rb:223:in `new'
>         from /usr/share/vagrant/plugins/kernel_v2/config/vm.rb:223:in
> `provision'
>         from /home/stuser/jmx/metrics-kafka/Vagrantfile:29:in `block (2
> levels) in <top (required)>'
>         from /usr/lib/ruby/vendor_ruby/vagrant/config/v2/loader.rb:37:in
> `call'
>         from /usr/lib/ruby/vendor_ruby/vagrant/config/v2/loader.rb:37:in
> `load'
>         from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:104:in
> `block (2 levels) in load'
>         from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:98:in
> `each'
>         from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:98:in
> `block in load'
>         from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:95:in
> `each'
>         from /usr/lib/ruby/vendor_ruby/vagrant/config/loader.rb:95:in
> `load'
>         from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:335:in
> `machine'
>         from /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:142:in
> `block in with_target_vms'
>         from /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:175:in
> `call'
>         from /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:175:in
> `block in with_target_vms'
>         from /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:174:in
> `map'
>         from /usr/lib/ruby/vendor_ruby/vagrant/plugin/v2/command.rb:174:in
> `with_target_vms'
>         from /usr/share/vagrant/plugins/commands/up/command.rb:56:in `block
> in execute'
>         from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:210:in `block
> (2 levels) in batch'
>         from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:208:in `tap'
>         from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:208:in `block
> in batch'
>         from <internal:prelude>:10:in `synchronize'
>         from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:207:in
> `batch'
>         from /usr/share/vagrant/plugins/commands/up/command.rb:55:in
> `execute'
>         from /usr/lib/ruby/vendor_ruby/vagrant/cli.rb:38:in `execute'
>         from /usr/lib/ruby/vendor_ruby/vagrant/environment.rb:484:in `cli'
>         from /usr/bin/vagrant:127:in `<main>'
>
> Any idea to make it work?
>
> thanks
>
>
> --
>
> Alec Li
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message