spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: Spark Streaming - how to implement multiple calculation using the same data set
Date Wed, 03 Sep 2014 02:05:17 GMT
Hi,

On Wed, Sep 3, 2014 at 6:54 AM, salemi <alireza.salemi@udo.edu> wrote:

> I was able to calculate the individual measures separately and know I have
> to merge them and spark streaming doesn't support outer join yet.
>

Can't you assign some dummy key (e.g., index) before your processing and
then join on that key using a function from
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.streaming.dstream.PairDStreamFunctions
?

Tobias

Mime
View raw message