hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wang Haihua (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12812) Enable mapred.input.dir.recursive by default to support union with aggregate function
Date Fri, 15 Jun 2018 18:10:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16514191#comment-16514191
] 

Wang Haihua commented on HIVE-12812:
------------------------------------

[~kgyrtkirk] Thanks for your reply.

 

For compability, we cannot idnetified if we set this parameter in hive-site.xml, which influence
of tez execution is, on the condition of tez also use FileInputformat for retriving data.

So i saw this would be a good solution which seperated and protected tez engine in our production.

 

For hive version, we are glad to update to 3.0, but for upgrading hive 2.1.1 which i have
done before just be a long period work..

 

you can see this work in my blog(Sorry there has only chinese version)   [https://ericsahit.github.io/2017/09/10/Hive%E5%8D%87%E7%BA%A7%E5%85%A8%E5%A7%BF%E5%8A%BF/]

 

 

> Enable mapred.input.dir.recursive by default to support union with aggregate function
> -------------------------------------------------------------------------------------
>
>                 Key: HIVE-12812
>                 URL: https://issues.apache.org/jira/browse/HIVE-12812
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive
>    Affects Versions: 1.2.1, 2.1.0
>            Reporter: Chaoyu Tang
>            Priority: Major
>         Attachments: HIVE-12812.patch, HIVE-12812.patch, HIVE-12812.patch
>
>
> When union remove optimization is enabled, union query with aggregate function writes
its subquery intermediate results to subdirs which needs mapred.input.dir.recursive to be
enabled in order to be fetched. This property is not defined by default in Hive and often
ignored by user, which causes the query failure and is hard to be debugged.
> So we need set mapred.input.dir.recursive to true whenever union remove optimization
is enabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message