storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessio Pagliari <>
Subject Re: Storm throughput
Date Fri, 30 Mar 2018 15:24:32 GMT
Surely they work on a way more powerful cluster, but the topology is composed by just one spout.
No parallelization, no bolts, for a total of one worker, so 1 thread in a jvm. Even if I had
100 cores like them it shouldn't make any difference. Please, correct me if I'm wrong.

Such a topology will assign it's only spout to a worker in a node: so, the multi-node cluster
is pointless. Meanwhile, regarding the number of cores, one executor cannot be at the same
time on multiple cores, not being a multi-thread process. 

Is there some Storm or Java behavior that I'm not aware of?

Thank you,


⁣Sent from BlueMail ​

On Mar 30, 2018, 4:28 PM, at 4:28 PM, Jacob Johansen <> wrote:
>for their test, they were using 4 worker nodes (servers) each with
>for a total of 96vCores.
>Most laptops max out at 8vCores and are typically at 4-6vCores
>Jacob Johansen
>On Fri, Mar 30, 2018 at 9:18 AM, Alessio Pagliari
>> Hi everybody,
>> I’m trying to do some preliminary tests with storm, to understand how
>> it can go. Now I’m focusing on trying to understand which is his
>> throughput in terms of tuples per second. I saw the benchmark done by
>> guys at Hortonworks (ref: https://it.hortonworks.
>> com/blog/microbenchmarking-storm-1-0-performance/) and in the first
>> they reach a spout emission rate of 3.2 million tuples/s.
>> I tried to replicate the test, a simple spout that emits continuously
>> same string “some data”. Differently from them, I’m using Storm 1.1.1
>> the storm cluster is set up on my laptop, anyway I’m just testing one
>> not an entire topology, but if you think that more configuration
>> information are needed, just ask.
>> To compute the throughput I ask the total amount of tuples processed
>> the UI APIs each 10s and I subtract it by the previous measure to
>have the
>> amount of tuples int the last 10s. What the mathematics give to me is
>> something around 32k tuples/s.
>> I don’t think to be wrong saying that 32k is not even comparable to
>> million. Is there something that I’m missing? Is it normal this
>> Thank you for your help and for your time,
>> Alessio

View raw message