storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navin Ipe <navin....@searchlighthealth.com>
Subject Re: Why aren't storm topology send and receive buffers equal sized?
Date Fri, 27 Jan 2017 09:52:15 GMT
I found a part of the answer in the article itself "Note that
topology.receiver.buffer.size is in contrast to the other buffer size
related parameters described in this article actually not configuring the
size of an LMAX Disruptor queue. Rather it sets the size of a simple
ArrayList that is used to buffer incoming messages because in this specific
case the data structure does not need to be shared with other threads, i.e.
it is local to the worker’s receive thread."

But the question still remains. If Bolt A sends 1024 tuples, then won't it
overwhelm Bolt B?


On Fri, Jan 27, 2017 at 3:11 PM, Navin Ipe <navin.ipe@searchlighthealth.com>
wrote:

> Hi,
>
> When going through Michael Noll's explanation
> <http://www.michael-noll.com/blog/2013/06/21/understanding-storm-internal-message-buffers/>
> on the buffers, I noticed the topology.receive.buffer.size default is 8 and
> topology.send.buffer.size default is 1024.
>
> If I have a topology where Spout --> Bolt A ---> BoltB, then if Bolt A
> sends a tuple of size 1024, then BoltB won't be able to handle 1024 easily
> and will get overwhelmed because Bolt B's receive buffer size is just 8.
>
> Is my above understanding correct? Why are the buffers so different in
> sizes?
>
> --
> Regards,
> Navin
>



-- 
Regards,
Navin

Mime
View raw message