samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Maquet <nmaq...@gmail.com>
Subject Re: How do you monitor Samza?
Date Mon, 12 Sep 2016 17:56:03 GMT
Hey Tommy,

At Movio, we feed all of our Samza metrics into Prometheus and use Grafana
to create visual dashboards. We configure all of our jobs to produce
metrics into a single topic and we have a monitoring tool that consumes
that topic and exposes the data to Prometheus (
https://github.com/movio/samza-prometheus-exporter).

We've been using this setup in production for 6 months now and it's been
working really well for us.

Nick


On 13 Sep 2016 1:04 a.m., "Tommy Becker" <tobecker@tivo.com> wrote:

> One of the challenges we've faced with deploying Samza is effectively
> monitoring it. We currently use the MetricsSnapshotReporter and consume the
> resulting stream pulling out specific values that we want to monitor, but
> this approach isn't scalable. We're exploring pumping the metrics into an
> ELK stack, but the format isn't ideal (many keys contain dots, which ES
> doesn't like). Additionally, many of the keys are "dynamic" in that the
> names are determined by runtime configuration, which makes pulling specific
> values out a bit of a pain. I'd love to get some information on how other
> folks are monitoring their Samza jobs.
>
> --
> Tommy Becker
> Senior Software Engineer
>
> Digitalsmiths
> A TiVo Company
>
> www.digitalsmiths.com<http://www.digitalsmiths.com>
> tobecker@tivo.com<mailto:tobecker@tivo.com>
>
> ________________________________
>
> This email and any attachments may contain confidential and privileged
> material for the sole use of the intended recipient. Any review, copying,
> or distribution of this email (or any attachments) by others is prohibited.
> If you are not the intended recipient, please contact the sender
> immediately and permanently delete this email and any attachments. No
> employee or agent of TiVo Inc. is authorized to conclude any binding
> agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo
> Inc. may only be made by a signed written agreement.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message