storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ankur Garg <ankurga...@gmail.com>
Subject Multiple Spouts in Same topology or Topology per spout
Date Sun, 11 Oct 2015 16:07:39 GMT
Hi ,

So I have a situation where I want to read messages from different queues
hosted in a Rabbitmq Server .

Now , there are three ways which I can think to leverage Apache Storm here
:-

1) Use the same Spout (say Spout A) to read messages from different queues
and based on the messages received emit it to different Bolts.

2) Use different Spout (Spout A and Spout B and so on) within the same
topology (say Topology A) to read messages from different queues .

3) Use Different Spouts one within eachTopology (Topology A , Topology B
and so on) to read messages from different queues .

Which is the best way to process this considering I want high throughput
(more no of queue messages to be processed concurrently) .

Also , If In use same Topology for all Spouts (currently though requirement
is for 2 spouts)  will failure in one Spout (or its associated Bolts)
effect the second or will they both continue working separately even if
some failure is in Spout B ?

Cost wise , how much would it be to maintain two different topologies .

Looking for inputs from members here.

Thanks
Ankur

Mime
View raw message