spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Join RDDs with DStreams
Date Fri, 09 Jan 2015 07:33:41 GMT
Here's how you do it:

val joined_stream = *myStream*.transform((x: RDD[(String, String)]) =>
{          val prdd = new PairRDDFunctions[String, String](x)
prdd.join(*myRDD*)        })




Thanks
Best Regards

On Thu, Jan 8, 2015 at 10:20 PM, Asim Jalis <asimjalis@gmail.com> wrote:

> Is there a way to join non-DStream RDDs with DStream RDDs?
>
> Here is the use case. I have a lookup table stored in HDFS that I want to
> read as an RDD. Then I want to join it with the RDDs that are coming in
> through the DStream. How can I do this?
>
> Thanks.
>
> Asim
>

Mime
View raw message