storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ankur Garg <>
Subject Multiple Spouts in Same topology or Topology per spout
Date Sun, 11 Oct 2015 16:07:39 GMT
Hi ,

So I have a situation where I want to read messages from different queues
hosted in a Rabbitmq Server .

Now , there are three ways which I can think to leverage Apache Storm here

1) Use the same Spout (say Spout A) to read messages from different queues
and based on the messages received emit it to different Bolts.

2) Use different Spout (Spout A and Spout B and so on) within the same
topology (say Topology A) to read messages from different queues .

3) Use Different Spouts one within eachTopology (Topology A , Topology B
and so on) to read messages from different queues .

Which is the best way to process this considering I want high throughput
(more no of queue messages to be processed concurrently) .

Also , If In use same Topology for all Spouts (currently though requirement
is for 2 spouts)  will failure in one Spout (or its associated Bolts)
effect the second or will they both continue working separately even if
some failure is in Spout B ?

Cost wise , how much would it be to maintain two different topologies .

Looking for inputs from members here.


View raw message