storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Xor <andreas.gramme...@gmail.com>
Subject Re: tuple size constraints?
Date Mon, 04 Jan 2016 10:12:22 GMT
Hello,

 TL;DR That depends on your hardware and configuration.

The slightly longer explanation being, that since the tuples are stored in
queues you have to keep consuming the queues at least to the rate that they
are fed. So, if your hardware can't process/send the tuples at the required
rate, probably problems will occur. Now on the large tuple size thing
depends again on the rate on which you send your tuples, say for example if
you send 1MB / sec or similar probably you won't have any problems...
assuming the network stack you use is up to the task. Although, if you want
to send many tuples / sec that are of large size you will need to have a
beast configuration to keep with the rates -- don't forget that each packet
has both I/O cost as well as CPU costs.

Hope this helped.


<https://www.dropbox.com/s/yxvycjvlsc111bh/pgpsig.txt>
On Mon, Jan 4, 2016 at 11:54 AM, Matan Safriel <dev.matan@gmail.com> wrote:

> Have some people here had experience with certain large sizes, which they
> care to report?
>
> On Fri, Jan 1, 2016 at 10:05 PM, anshu shukla <anshushukla0@gmail.com>
> wrote:
>
>> As per  my knowledge there is no limitation on tuple size and large tuple
>> sizes impact both processing and communication time (increasing the queuing
>> delay) .
>>
>> On Sat, Jan 2, 2016 at 1:19 AM, Matan Safriel <dev.matan@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I find the mailing list a little harder to search than the phased out
>>> google group, so sorry if this is a repeat of anything.
>>> I would like to learn whether Storm imposes any limitation on the size
>>> of tuples, or whether you have any comment about long tuples in Storm
>>> (version 0.10 and foreseeable roadmap).
>>>
>>> On the same note, are all tuples (regardless of size) cached until they
>>> have been successfully processed?
>>>
>>> Pointers to the appropriate code sections or plain answers are
>>> appreciated.
>>>
>>> Thanks,
>>> Matan
>>>
>>>
>>
>>
>> --
>> Thanks & Regards,
>> Anshu Shukla
>>
>
>

Mime
View raw message