hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From László Bodor (Jira) <>
Subject [jira] [Updated] (HIVE-21954) QTest: support for running qtests on various metastore DBs
Date Fri, 01 Nov 2019 15:33:00 GMT


László Bodor updated HIVE-21954:
    Attachment: HIVE-21954.07.patch

> QTest: support for running qtests on various metastore DBs
> ----------------------------------------------------------
>                 Key: HIVE-21954
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Standalone Metastore, Testing Infrastructure
>            Reporter: László Bodor
>            Assignee: László Bodor
>            Priority: Major
>             Fix For: 4.0.0
>         Attachments: HIVE-21954.01.patch, HIVE-21954.02.patch, HIVE-21954.03.patch, HIVE-21954.03.patch,
HIVE-21954.03.patch, HIVE-21954.04.patch, HIVE-21954.05.patch, HIVE-21954.07.patch, HIVE-21954.07.patch
> In HIVE-21940, a postgres metastore related issue has been fixed, and a local reproduction
has been provided.
> {code}
> export QTEST_LEAVE_FILES=true
> docker kill metastore-test-postgres-install
> docker rm metastore-test-postgres-install
> cd standalone-metastore
> mvn verify -DskipITests=false -Dit.test=ITestPostgres#install -Dtest=nosuch
> cd ..
> mvn test -Dtest.output.overwrite=true -Pitests,hadoop-2 -pl itests/qtest -Dtest=TestCliDriver
-Dqfile=partition_params_postgres.q -Dhive.metastore.rawstore.impl=org.apache.hadoop.hive.metastore.ObjectStore
> {code}
> The problem with this solution is that data/conf/hive-site.xml has to be edited manually.
My proposal is to introduce a property (-Dmetastore.db=postgres), which can take care of the
parameters on the fly. 2 supported solutions could be:
> 1. simple parameters: -Dmetastore.db=postgres
> In this case, tests depend on settings from ITestPostgres class (password, db, etc.)
> 2. verbose but flexible parameters: [see hive-site.xml HIVE-21940's repro patch|]

> In the first implementation, I would not start metastore db automatically (which is done
be 'mvn verify ...'), but it's still under planning. 
> In the long term, we should consider running this kind of tests in precommit phase, so
maybe -Dmetastore.db=postgres could start metastore db automatically. Also we should consider
running some qtests on various metastores. I would not pick randomly, but choose some "metastore-heavy"
ones instead.

This message was sent by Atlassian Jira

View raw message