spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lalitha MV <lalitham...@gmail.com>
Subject Re: Enforcing shuffle hash join
Date Mon, 04 Jul 2016 19:23:45 GMT
Hi maropu,

Thanks for your reply.

Would it be possible to write a rule for this, to make it always pick
shuffle hash join, over other join implementations(i.e. sort merge and
broadcast)?

Is there any documentation demonstrating rule based transformation for
physical plan trees?

Thanks,
Lalitha

On Sat, Jul 2, 2016 at 12:58 AM, Takeshi Yamamuro <linguin.m.s@gmail.com>
wrote:

> Hi,
>
> No, spark has no hint for the hash join.
>
> // maropu
>
> On Fri, Jul 1, 2016 at 4:56 PM, Lalitha MV <lalithamv92@gmail.com> wrote:
>
>> Hi,
>>
>> In order to force broadcast hash join, we can set
>> the spark.sql.autoBroadcastJoinThreshold config. Is there a way to enforce
>> shuffle hash join in spark sql?
>>
>>
>> Thanks,
>> Lalitha
>>
>
>
>
> --
> ---
> Takeshi Yamamuro
>



-- 
Regards,
Lalitha

Mime
View raw message