Something is definitely broken in your run or in your measurement method.... and its not your hardware that is at fault. The machine on which those numbers were run had lots of cores but the cores were not fast at all. Even my mid 2015 macbook pro has faster cores than that machine which had old Intel CPUs.
You maybe making some mistakes in your calculations. Just run the topo for about 14 mins and take the 10 min window reading directly from the UI and calculate the per sec throughput from that. (that way you disregard the first 3 or 4mins to allow for warm up). Also are you overriding any default settings ?
On Friday, March 30, 2018, 8:24:39 AM PDT, Alessio Pagliari <firstname.lastname@example.org> wrote:
Surely they work on a way more powerful cluster, but the topology is composed by just one spout. No parallelization, no bolts, for a total of one worker, so 1 thread in a jvm. Even if I had 100 cores like them it shouldn't make any difference. Please, correct me if I'm wrong.
Such a topology will assign it's only spout to a worker in a node: so, the multi-node cluster is pointless. Meanwhile, regarding the number of cores, one executor cannot be at the same time on multiple cores, not being a multi-thread process.
Is there some Storm or Java behavior that I'm not aware of?
On Mar 30, 2018, at 4:28 PM, Jacob Johansen <email@example.com
for their test, they were using 4 worker nodes (servers) each with 24vCores for a total of 96vCores.
Most laptops max out at 8vCores and are typically at 4-6vCores