storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stepan Urban <stepann.ur...@gmail.com>
Subject Re: LoggingMetricsConsumer stop logging
Date Wed, 07 Oct 2015 06:55:31 GMT
Hi Abhishek,
no bolt is not sending tupes on the metrics strem. Number of emitted and
transfered tuples is 340 and is not inceasing.

Thanks
Stepan

2015-10-07 7:16 GMT+02:00 Abhishek Agarwal <abhishcool@gmail.com>:

> Can you check on UI that the bolt is sending tuples on the metric stream?
> You have to click on "show system stats" on UI.
> On Oct 6, 2015 2:34 PM, "Stepan Urban" <stepann.urban@gmail.com> wrote:
>
>> Hi,
>> I tried to increase parallelism of bolts and create some performance
>> optimizations with no effect. Metrics stop logging after few hours. Is
>> there some mechanism that Storm stops logging when bolts are overloaded?
>> Any other sugesstions?
>>
>> Thanks
>> Stepan
>>
>> 2015-10-01 19:39 GMT+02:00 Stepan Urban <stepann.urban@gmail.com>:
>>
>>> 1. All metrics are not being logged
>>> 2. Yes topology is still prgressing, everything other works as normal,
>>> worker logs are still running
>>> 3. Bolt is slower, it writes data to disk, do some simple preprocessing
>>> and send them to another bolt, spout only receive it and send it to bolt.
>>>
>>> Thanks
>>> Stepan
>>>
>>> 2015-10-01 13:19 GMT+02:00 Abhishek Agarwal <abhishcool@gmail.com>:
>>>
>>>> Please check the following -
>>>> 1. Are all the metrics not being logged or is it just the custom
>>>> metrics? Are the in-built system metrics being logged?
>>>> 2. When the metrics are not getting logged, is the topology still
>>>> progressing?
>>>> 3. Is your bolt too slow compared to the spout?
>>>>
>>>> On Thu, Oct 1, 2015 at 3:14 PM, Stepan Urban <stepann.urban@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Ahishek,
>>>>> thanks for the response. Registring metrics is normal:
>>>>> transient MultiCountMetric countMetric;
>>>>> transient MultiReducedMetric meanMetric;
>>>>>
>>>>> countMetric = new MultiCountMetric();
>>>>> meanMetric = new MultiReducedMetric(new MeanReducer());
>>>>> context.registerMetric("count", countMetric, 1);
>>>>> context.registerMetric("mean", meanMetric, 1);
>>>>>
>>>>> and
>>>>> Config conf = new Config();
>>>>> conf.registerMetricsConsumer(LoggingMetricsConsumer.class, 1);
>>>>>
>>>>> even with default cluster.xml it stops logging on production computer.
>>>>>
>>>>> Yesterday I found that everything work on dev computer with data
>>>>> loaded from file. But on production computer, when data are receided
from
>>>>> net it stops logging after few hours. Is it possible to find if some
thread
>>>>> is not working properly using jstack? Or any other way how to debug it?
>>>>>
>>>>> Thanks
>>>>> Stepan
>>>>>
>>>>>
>>>>>
>>>>> 2015-10-01 6:26 GMT+02:00 Anishek Agarwal <anishek@gmail.com>:
>>>>>
>>>>>> From what i have found dont remove/add appenders in cluster.xml.
not
>>>>>> sure about why it works for a few hours and then stops. any specific
way
>>>>>> you are registering log metrics, is it possible to put some code
out ?
>>>>>>
>>>>>> On Tue, Sep 29, 2015 at 9:14 PM, Stepan Urban <
>>>>>> stepann.urban@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> I am using LoggingMetricsConsumer and MultiReducedMetric to count
>>>>>>> several metrics in Storm topology. After few hours of running
it stops
>>>>>>> logging these metrics into metrics.log file without an error
in any log
>>>>>>> file. Any sugestions how to find where is the problem? First
I used default
>>>>>>> cluster.xml with increased size of files, now I use only
>>>>>>> ch.qos.logback.core.FileAppender with the same results. After
few mb of
>>>>>>> logs it stops logging.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Stepan
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Abhishek Agarwal
>>>>
>>>>
>>>
>>

Mime
View raw message