spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Ash <and...@andrewash.com>
Subject .intersection() method on RDDs?
Date Fri, 24 Jan 2014 01:18:34 GMT
Hi spark users,

I recently wanted to calculate the set intersection of two RDDs of Strings.
 I couldn't find a .intersection() method in the autocomplete or in the
Scala API docs, so used a little set theory to end up with this:

lazy val A = ...
lazy val B = ...
A.union(B).subtract(A.subtract(B)).subtract(B.subtract(A))

Which feels very cumbersome.

Does anyone have a more idiomatic way to calculate intersection?

Thanks!
Andrew

Mime
View raw message