storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Webb" <Dave.W...@gmx.de>
Subject Metrics Period
Date Sat, 29 Oct 2016 14:20:53 GMT
Hi,

I'm experimenting with Storm Metrics and stumbled upon some strange behavior.
For some testing, I registered a CountMetric with a very short bucket time of 1 seconds:

        context.registerMetric("count", countMetric, 1);

First, I ran the topology with *no* additional "OutputBolt", so all tuples are eventually
discarded.
With this setup, my MetricsConsumer receives the Metric very regularly and the measured "count"
was high.

...
2016-10-29 15:26:51.366 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:26:52.568 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:26:53.644 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:26:54.803 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
...
-> very regularly every 1 sec

Then, I added an additional "OutputBolt" which stores the final tuples in a database. Obviously,
this causes
the measured "count" to significantly decrease, which I expected.
However, this setup also causes my MetricsConsumer to receive Updates *much* less regularly:

...
2016-10-29 15:39:52.555 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:38:29.732 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:39:13.162 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
2016-10-29 15:39:45.867 s.m.m.CustomMetricsConsumer [INFO] Received Metrics
...
-> not regularly: only every 30-45 seconds

I see that the load on the topology is high and it seems to "fill up" all queues, but it doesn't
seem right
that the Metric consumption gets this bad so quickly. The machine itself has plenty of resources
left to handle
proper Metric reporting.

Are there better way to receive Metrics with a higher frequency (like every 1-5 seconds)?

Mime
View raw message