Yes, it is practical and possible to build topology dynamically.

You can create a configuration file containing the details about the spout/bolts, grouping scheme, parallelisms etc. Then you write a program to parse this configuration and use the storm properties to set all the required details and then you can dynamically build the topology and submit it to Nimbus.


I have done a similar thing in our project. Where in one can define the topology configuration in the xml file and we read that configuration and build the topology from it.





From: Nathan Leung [mailto:ncleung@gmail.com]
Sent: Wednesday, December 03, 2014 9:03 AM
To: user; yuhao
Subject: Re: Is it possible to build topology dynamically


Not yet but I am working on it.  You can track progress here: https://issues.apache.org/jira/browse/STORM-561


On Tue, Dec 2, 2014 at 8:54 PM, yuhao <yuhao@easyway.net.cn> wrote:



Is it possible to build topology dynamically? For example, given some topology configuration which can tell names of spouts and bolts, as well as their parallelisms and how they are connected, the grouping scheme and the like, the program can then parse the configuration, build the topology and submit it to Nimbus.

Is this practical?


Best regards,

Hao Yu
Center for High-speed Railway Technology, Tsinghua University (CHRT)
Addr: 1117 LiuQing Building, Tsinghua University, Beijing, P.R.CHINA, 100084
Fax: 86-10-62788330