spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject How to compute a net (difference) given a bi-directional stream of numbers using spark streaming?
Date Wed, 24 Aug 2016 21:48:04 GMT
Hi Guys,
I am new to spark but I am wondering how do I compute the difference given a
bidirectional stream of numbers using spark streaming? To put it more concrete
say Bank A is sending money to Bank B and Bank B is sending money to Bank A
throughout the day such that at any given time we want to compute the difference
to find out who owe's who and how much?
The Algorithm can simply be breaking the stream into small intervals (which
spark streaming looks like it does) and then computing the difference in each
time interval and summing the results of each time interval (assuming the
numbers in one interval doesn't overlap with other intervals).
I would like to know what is the best way to achieve this using spark streaming?
I am also open to any other suggestions in the Algorithm.
Thanks!
Mime
View raw message