Hi Nathan,

You are right, I guess that was the case.

But I am not sure why even after increasing the spouts to 20 I couldn't end up with max throughput, I hold 4 core Xeon processor Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz, although I could see the high CPU utilization reaching more than 350% but is it the only reason behind the less performance gain than expected or do I need to add some configuration tuning.

Thanks!

On Wed, Feb 25, 2015 at 5:53 PM, Nathan Leung <ncleung@gmail.com> wrote:
It looks like you configured your topology to only use one worker process.

On Wed, Feb 25, 2015 at 6:39 AM, Vineet Mishra <clearmidoubt@gmail.com> wrote:
Hi All,

I am running a 3 node storm cluster with a Topology which is subscribed to Kafka using Kafka-Storm Spout.

The Cluster is hosting Nimbus on a node and left with 2 other nodes for the workers and supervisor to function.

While running the Topology I could see that all the spouts(20) and bolts(5) tasks are running on the same machine leaving the other node instance without any CPU utilization.

[1-1] 31m 7s ip-20-0-0-75 6703 2264180 2264180 11.519 2262620 0
[2-2] 31m 7s ip-20-0-0-75 6703 841540 841540 19.717 840940 0
[3-3] 31m 7s ip-20-0-0-75 6703 1080080 1080080 18.105 1079240 0
[4-4] 31m 7s ip-20-0-0-75 6703 1113900 1113900 17.966 1105320 0
[5-5] 31m 7s ip-20-0-0-75 6703 854220 854220 19.342 856240 0
[6-6] 31m 7s ip-20-0-0-75 6703 1249640 1249640 17.071 1251320 0
[7-7] 31m 7s ip-20-0-0-75 6703 2266520 2266520 11.63 2267260 0
[8-8] 31m 7s ip-20-0-0-75 6703 629760 629760 20.8 629120 0
[9-9] 31m 7s ip-20-0-0-75 6703 854360 854360 19.513 851800 0
[10-10] 31m 7s ip-20-0-0-75 6703 1417380 1417380 15.78 1418200 0
[11-11] 31m 7s ip-20-0-0-75 6703 858620 858620 20.897 858180 0
[12-12] 31m 7s ip-20-0-0-75 6703 1242980 1242980 16.766 1241100 0
[13-13] 31m 7s ip-20-0-0-75 6703 352300 352300 19.975 349420 0
[14-14] 31m 7s ip-20-0-0-75 6703 627920 627920 20.329 625440 0
[15-15] 31m 7s ip-20-0-0-75 6703 1061880 1061880 18.263 1061300 0
[16-16] 31m 7s ip-20-0-0-75 6703 706480 706480 20.297 702720 0
[17-17] 31m 7s ip-20-0-0-75 6703 1518800 1518800 15.154 1514640 0
[18-18] 31m 7s ip-20-0-0-75 6703 622640 622640 20.743 626240 0
[19-19] 31m 7s ip-20-0-0-75 6703 3376960 3376960 8.361 3377060 0
[20-20] 31m 7s ip-20-0-0-75 6703 1044940 1044940 18.569 1043760 0

Same is the case for bolts,

[22-22]} 32m 13s ip-20-0-0-75 6703 0 0 0.002 0.05 1376560 0.042 1376540 0
[23-23]} 32m 13s ip-20-0-0-75 6703 0 0 0.001 0.021 347180 0.02 347180 0
[24-24]} 32m 13s ip-20-0-0-75 6703 0 0 0 0.023 34600 0.023 34600 0
[25-25]} 32m 13s ip-20-0-0-75 6703 0 0 0 0.017 8380 0.014 8380 0
[26-26]} 32m 13s ip-20-0-0-75 6703 0 0 0.082 0.039 22217940 0.041 22217920 0

ip-20-0-0-75 is only utilized while their is another host which is not utilized at all. Please find the supervisor summary below

8e550ab9-9e0d-48a7-8899-0963d5f00a41 ip-20-0-0-78 1d 5h 7m 32s 4 0
9d98d0b4-1bb4-42e9-9a72-a67b82c64b2c ip-20-0-0-75 5d 23h 57m 53s 4 1
This is not the regular behavior of Storm as I have seen both the nodes operating while running the jobs, but how we can comment on the stability of the storm with this kind off behavior.

Urgent Issue, Request for expert intervention.

Thanks!