spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerry Raj <>
Subject Spark SQL DSL for joins?
Date Wed, 17 Dec 2014 05:43:01 GMT
I'm using the Scala DSL for Spark SQL, but I'm not able to do joins. I 
have two tables (backed by Parquet files) and I need to do a join across 
them using a common field (user_id). This works fine using standard SQL 
but not using the language-integrated DSL neither

t1.join(t2, on = 't1.user_id == t2.user_id)


t1.join(t2, on = Some('t1.user_id == t2.user_id))

work, or even compile. I could not find any examples of how to perform a 
join using the DSL. Any pointers will be appreciated :)


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message