storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@apache.org>
Subject Re: Storm Scheduler
Date Thu, 07 Jan 2016 09:00:34 GMT
What do you mean by "need 2 workers"? The parameter
"Config.setNumWorkers()"?

According to the code, the first submitted topology is getting 2 workers
and the second one 1 worker, assuming that the submission does trigger
the scheduler after each other (and as long as no re-scheduling happens),

https://github.com/apache/storm/blob/master/storm-core/src/clj/backtype/storm/scheduler/DefaultScheduler.clj

If, for example both topologies run into an error and a worker crashes
for each, it might happen that the second topology get re-scheduled
first and now it gets two worker slots.

In general, Storm does not give any ordering guarantees as the order of
topologies in parameter `Topologies` is "random".

Code always reviles the magic ;)

-Matthias

On 01/06/2016 10:00 PM, Rudraneel chakraborty wrote:
> Hi
> 
> Can anyone tell me , how storm schedules multiple topologies? 
> 
> For an example if topology A and B both need 2 workers (in total 4 ) but
> only 3 worker solots available , which topology will get 2 worker and
> which will get one worker? 
> 
> I know the default scheduler tries to evenly distribute executors to
> workers but i am interested in topology level scheduling
> 
> My experience says that its a fifo order , the topology submitted first
> will be always scheduled first 
> 
> On Tuesday, 8 December 2015, Rudraneel chakraborty
> <rudraneel.chakraborty@gmail.com
> <mailto:rudraneel.chakraborty@gmail.com>> wrote:
> 
>     Hello ,
> 
>     I am trying to develop a custom storm scheduler for some specific
>     needs. I wanted to know if one can control the calling of the
>     scheduler ( or, if it is called periodically , how to control the
>     frequency )
> 
>     -- 
>     Rudraneel Chakraborty
> 
> 
> 
> -- 
> Rudraneel Chakraborty
> Carleton University Real Time and Distributed Systems Reserach
> 


Mime
View raw message