flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kurt Young (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-15498) Using HiveCatalog in TPC-DS e2e
Date Tue, 07 Jan 2020 10:20:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009589#comment-17009589
] 

Kurt Young edited comment on FLINK-15498 at 1/7/20 10:19 AM:
-------------------------------------------------------------

Reusing an e2e test sounds really fragile to me. We might modify the e2e tests for various
reasons, there is no guarantee for users that the e2e test will serve like a benchmark tool
forever. 


was (Author: ykt836):
Reusing a e2e test sounds really fragile to me. We might modify the e2e tests for various
reasons, there are no guarantee for users that the e2e test will serve like a benchmark tools
forever. 

> Using HiveCatalog in TPC-DS e2e
> -------------------------------
>
>                 Key: FLINK-15498
>                 URL: https://issues.apache.org/jira/browse/FLINK-15498
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / Planner, Tests
>            Reporter: Jingsong Lee
>            Priority: Major
>             Fix For: 1.11.0
>
>
> In 1.10, we have made great progress in the performance and function of batch. After
our internal test, the performance is significantly ahead of hive.
> But it's hard for users to reproduce. They need to have some research on TPC-DS to write
test code.
> We can consider changing the E2E test of TPC-DS to HiveCatalog, which is roughly divided
into two stages:
>  # The first stage is prepare of hive. Prepare the tables of TPC-DS. Insert the data
and prepare the metastore. And analysis the tables.
>  # The second stage is the analysis of Flink. Only select and check results.
> Users can play with it only by changing the data scale of the first stage.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message