storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bishnu Patro <Bishnu_Pa...@symantec.com>
Subject RE: RE: Is it possible to build topology dynamically
Date Wed, 10 Dec 2014 04:02:46 GMT
Hi,

   Just have the topology configuration code to read from configuraton file in a java project
and include a class which contains the main function.
Inside the main function, code to build the topology and submit the topology to nimbus .

Also make sure that you deploy the jar inside the storm lib folder and from there, just invoke
your main function as part of deployment scripts.

Example:
[storm-lib]# /storm/bin/storm jar <<your jar file name>>.jar  <<package
path to the class including the class name containing main function>>  <<TopologyConfiguration
File Path>>

Regards
Bishnu

From: yuhao [mailto:yuhao@easyway.net.cn]
Sent: Wednesday, December 10, 2014 8:00 AM
To: Bishnu Patro
Subject: Re: RE: Is it possible to build topology dynamically

HI,

I thought about implementing it in Java, but stuck in building the topology into a jar which
can then be submitted to Nimbus.
Given the configuration, I can build the topology by calling addSpout(), addBolt(), and grouping...
apis. Then the topology codes
should be put into a jar, to do that, these codes need compiled as class files first - but
how to get this done in Java in a smart way?

________________________________
Best regards,

Hao Yu
Center for High-speed Railway Technology, Tsinghua University (CHRT)
Addr: 1117 LiuQing Building, Tsinghua University, Beijing, P.R.CHINA, 100084
Fax: 86-10-62788330
Email:yuhao@easyway.net.cn<mailto:yuhao@easyway.net.cn>

From: Bishnu Patro<mailto:Bishnu_Patro@symantec.com>
Date: 2014-12-09 16:44
To: user@storm.apache.org<mailto:user@storm.apache.org>; yuhao<mailto:yuhao@easyway.net.cn>
Subject: RE: Is it possible to build topology dynamically
Hi,

Yes, it is practical and possible to build topology dynamically.
You can create a configuration file containing the details about the spout/bolts, grouping
scheme, parallelisms etc. Then you write a program to parse this configuration and use the
storm properties to set all the required details and then you can dynamically build the topology
and submit it to Nimbus.

I have done a similar thing in our project. Where in one can define the topology configuration
in the xml file and we read that configuration and build the topology from it.

Regards
Bishnu

From: Nathan Leung [mailto:ncleung@gmail.com]
Sent: Wednesday, December 03, 2014 9:03 AM
To: user; yuhao
Subject: Re: Is it possible to build topology dynamically

Not yet but I am working on it.  You can track progress here: https://issues.apache.org/jira/browse/STORM-561

On Tue, Dec 2, 2014 at 8:54 PM, yuhao <yuhao@easyway.net.cn<mailto:yuhao@easyway.net.cn>>
wrote:
Hi,

Is it possible to build topology dynamically? For example, given some topology configuration
which can tell names of spouts and bolts, as well as their parallelisms and how they are connected,
the grouping scheme and the like, the program can then parse the configuration, build the
topology and submit it to Nimbus.
Is this practical?

________________________________
Best regards,

Hao Yu
Center for High-speed Railway Technology, Tsinghua University (CHRT)
Addr: 1117 LiuQing Building, Tsinghua University, Beijing, P.R.CHINA, 100084
Fax: 86-10-62788330
Email:yuhao@easyway.net.cn<mailto:yuhao@easyway.net.cn>

Mime
View raw message