hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17543) Enable PerfCliDriver for HoS
Date Mon, 25 Sep 2017 22:14:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179866#comment-16179866
] 

Sahil Takiar commented on HIVE-17543:
-------------------------------------

[~hsubramaniyan], [~pvary], [~lirui] could you review?

Main things of note:

* I renamed the current {{TestPerfCliDriver}} to {{TestTezPerfCliDriver}} and created a new
{{TestSparkPerfCliDriver}}
* I set {{hive.auto.convert.join}} to {{true}} for the {{TestSparkCliDriver}} since thats
closer to what is run in production
* I had to make some changes to {{SparkCrossProductCheck}} and {{SparkWork}} to avoid some
flakiness in the tests
* There are two TPC-DS queries that I couldn't get to work with HoS - query14 and query64
- I'll file follow up tasks for fixing them
* I haven't gone through the explain plans of every single TPC-DS query for HoS, mainly because
that will take a really long time, but I plan to do it as a follow up task; committing this
now will give us better regression testing

> Enable PerfCliDriver for HoS
> ----------------------------
>
>                 Key: HIVE-17543
>                 URL: https://issues.apache.org/jira/browse/HIVE-17543
>             Project: Hive
>          Issue Type: Improvement
>          Components: Spark
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>         Attachments: HIVE-17543.1.patch, HIVE-17543.2.patch, HIVE-17543.3.patch, HIVE-17543.4.patch
>
>
> The PerfCliDriver contains .q files for TPC-DS queries. It doesn't actually run them,
but it does generate explains for them. It also tricks HMS into thinking its a 30 TB TPC-DS
dataset so that the explain plan triggers certain optimizations.
> Right now this only runs of Hive-on-Tez, we should enable it for HoS too.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message