Why not experimenting? :)

Assuming that you're already having Kafka Spout so pulling data in mongo will be occurred from Bolt (Trident operator).
Then it depends on how long your pull query elapsed for given signal, since pulling data from external data source is tend to contribute major latency. Your query, schema of the table, index, and others matter. So experimenting with your query and your table schema, and your infra only give you to valid answer.

- Jungtaek Lim (HeartSaVioR)

2016년 10월 7일 (금) 오후 2:14, Cheney Chen <tbcql1986@gmail.com>님이 작성:
Wondering if anyone has any clues?

On Wed, Oct 5, 2016 at 12:10 PM, Cheney Chen <tbcql1986@gmail.com> wrote:
Hi there, 

I'm using storm 1.0.1. I have user case like this, given one signal (received from kafka spout), fetch a bunch of users (in mongodb), then do stream processing.

I'd prefer trident since the lambda like language is so convenient. But don't have good idea about iterating through mongodb. By checking storm source code, there's good example for insert and update but not loop.

So questions are:
1. Is it a good idea to iterate mongo in storm stream processing?
2. If so, any good idea or example?
3. If not, why? 


--
Regards,
Qili Chen (Cheney)

E-mail: tbcql1986@gmail.com
MP: (+1) 4086217503



--
Regards,
Qili Chen (Cheney)

E-mail: tbcql1986@gmail.com
MP: (+1) 4086217503