storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Conaway <kevin.a.cona...@gmail.com>
Subject Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm
Date Sun, 01 May 2016 12:17:01 GMT
One thing I would like to see added (if not already present) is the ability
to register metrics that are not tied to a component.

As of now, the only non-component metrics are reported by the SystemBolt
pseudo-component which feels like a work-around.  It reports JVM level
metrics like GC time, heap size and other things that aren't associated
with a given component.

It would be great if application developers could expose similar metrics
like this for things like connection pools and other JVM wide objects that
aren't unique to a specific component.

I don't think this is possible now, is it?

On Wed, Apr 20, 2016 at 12:29 AM, Jungtaek Lim <kabhwan@gmail.com> wrote:

> Let me start sharing my thought. :)
>
> 1. Need to enrich docs about metrics / stats.
>
> In fact, I couldn't see the fact - topology stats are sampled by default
> and sample rate is 0.05 - from the docs when I was newbie of Apache
> Storm. It made me misleading and made me saying "Why there're difference
> between the counts?". I also saw some mails from user@ about same question.
> If we include this to guide doc that would be better.
>
> And Metrics document page
> <http://storm.apache.org/releases/1.0.0/Metrics.html> seems not well
> written. I think it has appropriate headings but lacks contents on each
> heading.
> It should be addressed, and introducing some external metrics consumer
> plugins (like storm-graphite <https://github.com/verisign/storm-graphite> from
> Verisign) would be great, too.
>
> 2. Need to increase sample rate or (ideally) no sampling at all.
>
> Let's postpone considering performance hit at this time.
> Ideally, we expect precision of metrics gets better when we increase
> sample rate. It affects non-gauge kinds of metrics which are counter, and
> latency, and so on.
>
> Btw, I would like to hear about opinions on latency since I'm not an
> expert.
> Storm provides only average latency and it's indeed based on sample rate.
> Do we feel OK with this? If not how much having also percentiles can help
> us?
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2016년 4월 20일 (수) 오전 10:55, Jungtaek Lim <kabhwan@gmail.com>님이 작성:
>
>> Hi Storm users,
>>
>> I'm Jungtaek Lim, committer and PMC member of Apache Storm.
>>
>> If you subscribed dev@ mailing list, you may have seen that recently
>> we're addressing the metrics feature on Apache Storm.
>>
>> For now, improvements are going forward based on current metrics feature.
>>
>> - Improve (Topology) MetricsConsumer
>> <https://issues.apache.org/jira/browse/STORM-1699>
>> - Provide topology metrics in detail (metrics per each stream)
>> <https://issues.apache.org/jira/browse/STORM-1719>
>> - (WIP) Introduce Cluster Metrics Consumer
>>
>> As I don't maintain large cluster for myself, I really want to collect
>> the any ideas for improving, any inconveniences, use cases of Metrics with
>> community members, so we're on the right way to go forward.
>>
>> Let's talk!
>>
>> Thanks in advance,
>> Jungtaek Lim (HeartSaVioR)
>>
>


-- 
Kevin Conaway
http://www.linkedin.com/pub/kevin-conaway/7/107/580/
https://github.com/kevinconaway

Mime
View raw message