storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navin Ipe <>
Subject How to let a topology know that it's time to stop?
Date Thu, 05 May 2016 04:13:37 GMT

I know Storm is designed to run forever. I also know about Trident's
technique of aggregation. But shouldn't Storm have a way to let bolts know
that a certain bunch of processing has been completed?

Consider this topology:
            |                  |--->Bolt-B
            |                  \--->Bolt-B
            |                  |--->Bolt-B
            |                  \--->Bolt-B

   - From Bolt-A to Bolt-B, it is a FieldsGrouping.
   - Spout emits only a few tuples and then stops emitting.
   - Bolt A takes those tuples and generates millions of tuples.

*Bolt-B accumulates tuples that Bolt A sends and needs to know when Spout
finished emitting. Only then can Bolt-B start writing to SQL.*

1. How can all Bolts B be notified that it is time to write to SQL?
2. After all Bolts B have written to SQL, how to know that all Bolts B have
completed writing?
3. How to stop the topology? I know of
localCluster.killTopology("HelloStorm"), but shouldn't there be a way to do
it from the Bolt?


View raw message