storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jungtaek Lim <kabh...@gmail.com>
Subject Re: How to Improve Storm Application's Throughput
Date Mon, 07 Aug 2017 10:20:52 GMT
According to statistics of topology, some of your Bolts are not keeping up
handling tuples. (capacity is close to or even over 1.0 from some bolts)

- increase parallelism for the component, or try to ensure locality between
near components
- look into details on statistic from each task in component and see
whether load is distributed evenly
- try to change shuffle grouping to local or shuffle grouping when
applicable
- you may want to combine some near bolts into one if needed

Hope this helps.

Thanks,
Jungtaek Lim (HeartSaVioR)

On Mon, 7 Aug 2017 at 11:58 AM 王 纯超 <wangchunchao@outlook.com> wrote:

> Hi,
>
>     I am now considering improve a Storm application's throughput because
> I find that the consumption speed of KafkaSpout is slower than the
> producing speed. And the lag gets larger and larger. Below is the bolt
> statistics. I tried to bring forward the tuple projection and filtering
> logic in a custom scheme with intention of reducing network traffic.
> However, after observation, things go contrary to my wishes. Am I going the
> wrong way? Are there any principles tuning Storm applications? Or could
> anyone give some suggestions for this specific case?
> ------------------------------
> wangchunchao@outlook.com
>

Mime
View raw message