spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Jay <bill.jaypeter...@gmail.com>
Subject Spark Streaming RDD transformation
Date Thu, 26 Jun 2014 18:40:50 GMT
Hi all,

I am current working on a project that requires to transform each RDD in a
DStream to a Map. Basically, when we get a list of data in each batch, we
would like to update the global map. I would like to return the map as a
single RDD.

I am currently trying to use the function *transform*. The output will be a
RDD of the updated map after each batch. How can I create an RDD from
another data structure such as Int, Map, ect. Thanks!

Bill

Mime
View raw message