storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aravind s <>
Subject Priority polling across multiple Kafka spouts
Date Thu, 15 Mar 2018 14:04:20 GMT

Is there a way where we could fetch records from multiple spouts at the
particular ratio and merge into a single stream for processing as such.
Some kind of priority processing from multiple spouts a single topology
listens to as such.

For example
I have a three Kafka streams p1, p2, p3 as such

For 100 slots of buffer

If there are records in p1 then fill the buffer
If the buffer space exists then fill the buffer from the records from p2
Else from p3.

Maybe specify as weights to each to each stream to be processed as such.

Please do let me know is there a way we could do it.

Aravind S

View raw message