ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jungtaek Lim <>
Subject Re: Not intuitive behavior of AMS Grafana query
Date Sat, 04 Jun 2016 03:51:00 GMT
Hi Aravindan,

Thanks for the answer. I guessed it might be intended and was correct.

Btw, AMS-Grafana query editor can be more flexible like what I've showed,
or can restrict user's query via validating user's query or asserting
response has only one metric.
I think it will make AMS-Grafana query more intuitive way. What do you
think, and which is more preferred way when you agree?

And I'm feeling that I need to tell background story of my question to let
you understand my intention.

I'm recently working on pushing Storm metrics to AMS to let Ambari users
store Storm metrics and configure dashboards without installing time-series
DB manually. Ambari already provides Storm Metrics Sink for this, but it
occurs topology performance issue it's disabled by default.
(I'm addressing that issue from Storm side, STORM-1699

In parallel, I'm playing with current Storm Metrics Sink, and regardless of
performance issue, I found metrics are not pushing to AMS at all. I'm
addressing AMBARI-16946 <> to
make it works, but it requires metrics to be pushed via task level. This
change is necessary because there was metric level mismatch between Storm
(task) and Storm Metrics Sink (component), and aggregation being done with
 sink is not correct, and technically can't be correct (topology can set
'parallelism' of the sink).

After this change, Storm metrics sink will push metrics by task level, but
users often want to see component level metrics which needs aggregation.
For example, when sink pushes 'component1.task1.metric1',
'component1.task2.metric1', and so on, users want to see aggregated metric
by 'component1.%.metric1'. If metric is counter type, users normally want
to apply summation on that, and if metric is latency type, users normally
want to apply average on that.

Collector API now responses a request with multiple metrics / hosts (though
AMBARI-16949 <> blocks
this for now) but it still requires caller side aggregation. I was
considering either support aggregation via AMS Grafana plugin or support
aggregation via Collector API (AMBARI-17027
<>), and give latter
thing a try for now.

Thanks for reading long story, and please comment and review if you can
cover this side. I'm including Siddharth Wagle and Sriharsha Chintalapani
to reviewer but they seems not available for now. So if someone want to
review my patches I'd be really appreciated.

Thanks again!
Jungtaek Lim (HeartSaVioR)

2016년 6월 4일 (토) 오전 2:04, Aravindan Vijayan <>님이

> Hi Jungtaek,
> The AMS-Grafana query editor was designed to support only 1 metric & At
> most 1 host value per entry. You can use a cloned version of templated
> dashboard like "System-Servers" to see metrics for multiple hosts, for
> Storm.
> However, the AMS API which is used by Grafana supports multiple metric
> names as well as multiple hostnames separated by comma.
> For example,
> http://
> <collector_host>:6188/ws/v1/timeline/metrics?metricNames=disk_free,disk_used&hostname=h1,h2&appId=HOST<
> >
> Regarding your query on aggregation, if any metric(s) is requested WITHOUT
> host, it is by default an aggregated value across all hosts. You can select
> the aggregation function in the drop down as well. By default, it is
> average. For example, all the graphs on Hbase-Home dashboard show
> aggregated metrics across all HBase hosts.
> Thanks!
> --
> Thanks and Regards,
> Aravindan Vijayan
> From: Jungtaek Lim
> Reply-To: "<>"
> Date: Thursday, June 2, 2016 at 8:06 PM
> To: "<>"
> Subject: Not intuitive behavior of AMS Grafana query
> Hi devs,
> I'm posting thread instead of filing an issue since I'm not sure it's a
> bug or by intention.
> While I'm playing with AMS Grafana integration with Ambari 2.2.2, I found
> that metric field accepts wildcard ('%25' for '%') and multiple metric
> names (',') and host accepts multiple host names (',') but graph is showing
> only one.
> Please refer this screenshot.
> [ams-grafana-plugin-not-intuitive-behavior.png]
> Actually multiple metrics are matched but it shows only first one. Please
> note that the graph is not showing aggregated values.
> If we modify js a bit, it shows multiple metrics. Please refer below.
> [ams-grafana-plugin-not-intuitive-behavior-after-js-modification.png]
> While using '%25' feels kind of hack, there's a way to match multiple
> metrics anyway, and origin result seems not intuitive.
> What do you think? Would we want to address plugin to show latter result?
> Or just restricting metric and hosts input to not containing wildcard and
> comma?
> And what do you think about providing aggregation between multiple series
> like sumSeries / averageSeries / minSeries / maxSeries on graphite? Storm
> metric changes need this feature to aggregate task level metrics by
> component.
> Thanks,
> Jungtaek Lim (HeartSaVioR)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message