storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessio Pagliari <pagli...@i3s.unice.fr>
Subject Storm throughput
Date Fri, 30 Mar 2018 14:18:46 GMT
Hi everybody,

I’m trying to do some preliminary tests with storm, to understand how far it can go. Now
I’m focusing on trying to understand which is his maximum throughput in terms of tuples
per second. I saw the benchmark done by the guys at Hortonworks (ref: https://it.hortonworks.com/blog/microbenchmarking-storm-1-0-performance/
<https://it.hortonworks.com/blog/microbenchmarking-storm-1-0-performance/>) and in the
first test they reach a spout emission rate of 3.2 million tuples/s. 

I tried to replicate the test, a simple spout that emits continuously the same string “some
data”. Differently from them, I’m using Storm 1.1.1 and the storm cluster is set up on
my laptop, anyway I’m just testing one spout not an entire topology, but if you think that
more configuration information are needed, just ask. 

To compute the throughput I ask the total amount of tuples processed to the UI APIs each 10s
and I subtract it by the previous measure to have the amount of tuples int the last 10s. What
the mathematics give to me is something around 32k tuples/s.

I don’t think to be wrong saying that 32k is not even comparable to 3.2 million. Is there
something that I’m missing? Is it normal this output?

Thank you for your help and for your time,

Alessio
Mime
View raw message