storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Yost <>
Subject Fan in problem: virtually all time spent in network I/O wait
Date Thu, 10 Dec 2015 02:31:39 GMT
Hi Everyone,

I have a large fan in within my topology where I go from 1000 Bolt A
executors to 50 Bolt B executors via fieldsGrouping.  When I profile via
jvisualvm, it shows that the Bolt A thread spends 99% of it's time in the
com.lmax.disruptor.BlockingWaitStrategy.waitFor method.

The topology details are as follows:

200 workers
20 KafkaSpout executors
1000 Bolt A executors
50  Bolt B executors

fieldsGrouping from Bolt A -> Bolt B because I am caching in Bolt B,
building up large Key/Value pairs for HFile import into HBase.

I am thinking if I add an extra bolt between Bolt A and Bolt B where I do a
localOrShuffleGrouping to go from 1000 -> 200 locally followed by
fieldsGrouping to go from 200 -> 50 will lessen Network I/O wait time.

Please confirm if this makes sense or if there are any other better ideas.



View raw message