spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject ShuffledHashJoin's selection criteria
Date Mon, 23 Apr 2018 19:57:16 GMT
Hi,

I've been reviewing the code of JoinSelection for ShuffledHashJoin and
can't understand how !RowOrdering.isOrderable(leftKeys) can be ever met for
the second case (copying the entire code for a quick look):

         if !conf.preferSortMergeJoin && canBuildRight(joinType) &&
canBuildLocalHashMap(right)
           && muchSmaller(right, left) ||
           !RowOrdering.isOrderable(leftKeys) =>

         if !conf.preferSortMergeJoin && canBuildLeft(joinType) &&
canBuildLocalHashMap(left)
           && muchSmaller(left, right) ||
           !RowOrdering.isOrderable(leftKeys) =>

How is !RowOrdering.isOrderable(leftKeys) possible in the second case? I
must be missing something...again :( Please help.

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski

Mime
View raw message