Message view | « Date » · « Thread » |
---|---|
Top | « Date » · « Thread » |
From | Bill Jay <bill.jaypeter...@gmail.com> |
Subject | Spark Streaming RDD transformation |
Date | Thu, 26 Jun 2014 18:40:50 GMT |
Hi all, I am current working on a project that requires to transform each RDD in a DStream to a Map. Basically, when we get a list of data in each batch, we would like to update the global map. I would like to return the map as a single RDD. I am currently trying to use the function *transform*. The output will be a RDD of the updated map after each batch. How can I create an RDD from another data structure such as Int, Map, ect. Thanks! Bill | |
Mime |
|
View raw message |