storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gireesh Ramji <>
Subject Re: [help]storm spout complete latency very high
Date Thu, 23 Feb 2017 05:09:03 GMT
Have you checked the receive queue population of your bolts? If the executor receive buffer
size is 1024, say, and those buffers are full (or even non empty), that will add to the complete
latency of the tuples in the buffer without having any impact on the execute latency of the
bolt. This gets compounded across the 40 bolt executors that you have so there are potentially
40k tuples for which the complete latency is much higher than you anticipate based on the
execute latency.
In reality since you have set max.spout.pending to 256, you probably have a situation where
each of your 40 bolt executors have 256 tuples in their buffer, so there are 10k tuples for
which the complete latency is amiss and this is contributing the discrepancy you see in the
spout complete latency.

      From: daxwang(王文军) <>
 To: user <> 
 Sent: Thursday, February 23, 2017 12:06 PM
 Subject: [help]storm spout complete latency very high
 #yiv3716946653 body {line-height:1.5;}#yiv3716946653 blockquote {margin-top:0px;margin-bottom:0px;margin-left:0.5em;}#yiv3716946653
div.yiv3716946653foxdiv20170223120554122386 {}#yiv3716946653 body {font-size:10.5pt;color:rgb(0,
0, 0);line-height:1.5;}

in my storm topology, spouts have very high complete latency 28794.477(ms),but  bolt capacity(last
10m) and execute latency not high,  i start backpressure with default settings. and  set
 topology.max.spout.pending=256, in my topology the size of one tuple is about 1024KB,  so
what's wrong?  help please.
in my topology:storm version: 1.0.2worker : 10worker mem: 1Gspout: 40 executorbolt: 10 executor

  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message