storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Conaway <>
Subject Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm
Date Sun, 01 May 2016 12:17:01 GMT
One thing I would like to see added (if not already present) is the ability
to register metrics that are not tied to a component.

As of now, the only non-component metrics are reported by the SystemBolt
pseudo-component which feels like a work-around.  It reports JVM level
metrics like GC time, heap size and other things that aren't associated
with a given component.

It would be great if application developers could expose similar metrics
like this for things like connection pools and other JVM wide objects that
aren't unique to a specific component.

I don't think this is possible now, is it?

On Wed, Apr 20, 2016 at 12:29 AM, Jungtaek Lim <> wrote:

> Let me start sharing my thought. :)
> 1. Need to enrich docs about metrics / stats.
> In fact, I couldn't see the fact - topology stats are sampled by default
> and sample rate is 0.05 - from the docs when I was newbie of Apache
> Storm. It made me misleading and made me saying "Why there're difference
> between the counts?". I also saw some mails from user@ about same question.
> If we include this to guide doc that would be better.
> And Metrics document page
> <> seems not well
> written. I think it has appropriate headings but lacks contents on each
> heading.
> It should be addressed, and introducing some external metrics consumer
> plugins (like storm-graphite <> from
> Verisign) would be great, too.
> 2. Need to increase sample rate or (ideally) no sampling at all.
> Let's postpone considering performance hit at this time.
> Ideally, we expect precision of metrics gets better when we increase
> sample rate. It affects non-gauge kinds of metrics which are counter, and
> latency, and so on.
> Btw, I would like to hear about opinions on latency since I'm not an
> expert.
> Storm provides only average latency and it's indeed based on sample rate.
> Do we feel OK with this? If not how much having also percentiles can help
> us?
> Thanks,
> Jungtaek Lim (HeartSaVioR)
> 2016년 4월 20일 (수) 오전 10:55, Jungtaek Lim <>님이 작성:
>> Hi Storm users,
>> I'm Jungtaek Lim, committer and PMC member of Apache Storm.
>> If you subscribed dev@ mailing list, you may have seen that recently
>> we're addressing the metrics feature on Apache Storm.
>> For now, improvements are going forward based on current metrics feature.
>> - Improve (Topology) MetricsConsumer
>> <>
>> - Provide topology metrics in detail (metrics per each stream)
>> <>
>> - (WIP) Introduce Cluster Metrics Consumer
>> As I don't maintain large cluster for myself, I really want to collect
>> the any ideas for improving, any inconveniences, use cases of Metrics with
>> community members, so we're on the right way to go forward.
>> Let's talk!
>> Thanks in advance,
>> Jungtaek Lim (HeartSaVioR)

Kevin Conaway

View raw message