storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacob Johansen <johansenj...@gmail.com>
Subject Re: Storm throughput
Date Fri, 30 Mar 2018 14:27:15 GMT
for their test, they were using 4 worker nodes (servers) each with 24vCores
for a total of 96vCores.
Most laptops max out at 8vCores and are typically at 4-6vCores

Jacob Johansen

On Fri, Mar 30, 2018 at 9:18 AM, Alessio Pagliari <pagliari@i3s.unice.fr>
wrote:

> Hi everybody,
>
> I’m trying to do some preliminary tests with storm, to understand how far
> it can go. Now I’m focusing on trying to understand which is his maximum
> throughput in terms of tuples per second. I saw the benchmark done by the
> guys at Hortonworks (ref: https://it.hortonworks.
> com/blog/microbenchmarking-storm-1-0-performance/) and in the first test
> they reach a spout emission rate of 3.2 million tuples/s.
>
> I tried to replicate the test, a simple spout that emits continuously the
> same string “some data”. Differently from them, I’m using Storm 1.1.1 and
> the storm cluster is set up on my laptop, anyway I’m just testing one spout
> not an entire topology, but if you think that more configuration
> information are needed, just ask.
>
> To compute the throughput I ask the total amount of tuples processed to
> the UI APIs each 10s and I subtract it by the previous measure to have the
> amount of tuples int the last 10s. What the mathematics give to me is
> something around 32k tuples/s.
>
> I don’t think to be wrong saying that 32k is not even comparable to 3.2
> million. Is there something that I’m missing? Is it normal this output?
>
> Thank you for your help and for your time,
>
> Alessio
>

Mime
View raw message