spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yogesh Vyas <informy...@gmail.com>
Subject Fwd: filtering in SparkR
Date Mon, 03 Oct 2016 06:58:57 GMT
Hi,

I have two SparkDataFrames, df1 and df2.
There schemas are as follows:
df1=>SparkDataFrame[id:double, c1:string, c2:string]
df2=>SparkDataFrame[id:double, c3:string, c4:string]

I want to filter out rows from df1 where df1$id does not match df2$id

I tried some expression: filter(df1,!(df1$id %in% df2$id)), but it does not
works.

Anybody could please provide me a solution for it?

Regards,
Yogesh

Mime
View raw message