hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From László Bodor (Jira) <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-22790) QTests seem to fail when using MySQL docker backend db
Date Tue, 18 Feb 2020 10:59:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038940#comment-17038940
] 

László Bodor edited comment on HIVE-22790 at 2/18/20 10:58 AM:
---------------------------------------------------------------

the issue could be that datanucleus thinks that mariadb is oracle, as INITIALLY DEFERRED constraints
are only available in oracle world

UPDATE: debugged, the root cause is that Datanucleus doesn't have a database adapter for mariadb,
and uses BaseDatastoreAdapter, which by default support initially deferred constraints:
https://github.com/datanucleus/datanucleus-rdbms/blob/master/src/main/java/org/datanucleus/store/rdbms/adapter/BaseDatastoreAdapter.java#L520

that's why it appends the string:
https://github.com/datanucleus/datanucleus-rdbms/blob/master/src/main/java/org/datanucleus/store/rdbms/key/ForeignKey.java#L363-L366

https://issues.apache.org/jira/browse/JDO-781



was (Author: abstractdog):
the issue could be that datanucleus thinks that mariadb is oracle, as INITIALLY DEFERRED constraints
are only available in oracle world

UPDATE: debugged, the root cause is that Datanucleus doesn't have a database adapter for mariadb,
and uses BaseDatastoreAdapter, which by default support initially deferred constraints:
https://github.com/datanucleus/datanucleus-rdbms/blob/master/src/main/java/org/datanucleus/store/rdbms/adapter/BaseDatastoreAdapter.java#L520

that's why it appends the string:
https://github.com/datanucleus/datanucleus-rdbms/blob/master/src/main/java/org/datanucleus/store/rdbms/key/ForeignKey.java#L363-L366



> QTests seem to fail when using MySQL docker backend db
> ------------------------------------------------------
>
>                 Key: HIVE-22790
>                 URL: https://issues.apache.org/jira/browse/HIVE-22790
>             Project: Hive
>          Issue Type: Bug
>          Components: Standalone Metastore, Testing Infrastructure
>            Reporter: Zoltan Chovan
>            Assignee: László Bodor
>            Priority: Major
>
> Running qtests for example with the following command:
>  
> {code:java}
> mvn test -Dtest.output.overwrite=true -Pitests -pl itests/qtest -Dtest=TestCliDriver
-Dqfile=partition_params_postgres.q -Dtest.metastore.db=mysql
> {code}
>  
>  
> Interestingly it seems to fail even when checking out the patch (HIVE-21954) that introduced
the dockerized testing.
>  
> {code}
> 2020-02-18T01:23:24,563 ERROR [main] DataNucleus.Datastore: An exception was thrown while
adding/validating class(es) : (conn=7) You have an error in your SQL syntax; check the
> manual that corresponds to your MariaDB server version for the right syntax to use near
'INITIALLY DEFERRED' at line 1
> java.sql.SQLSyntaxErrorException: (conn=7) You have an error in your SQL syntax; check
the manual that corresponds to your MariaDB server version for the right syntax to use nea
> r 'INITIALLY DEFERRED' at line 1
>         at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:243)
>         at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:164)
>         at org.mariadb.jdbc.MariaDbStatement.executeExceptionEpilogue(MariaDbStatement.java:244)
>         at org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:335)
>         at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:461)
>         at com.zaxxer.hikari.pool.ProxyStatement.execute(ProxyStatement.java:92)
>         at com.zaxxer.hikari.pool.HikariProxyStatement.execute(HikariProxyStatement.java)
>         at org.datanucleus.store.rdbms.table.AbstractTable.executeDdlStatement(AbstractTable.java:879)
>         at org.datanucleus.store.rdbms.table.TableImpl.createForeignKeys(TableImpl.java:522)
>         at org.datanucleus.store.rdbms.table.TableImpl.validateForeignKeys(TableImpl.java:468)
>         at org.datanucleus.store.rdbms.table.TableImpl.validateConstraints(TableImpl.java:394)
>         at org.datanucleus.store.rdbms.table.ClassTable.validateConstraints(ClassTable.java:3576)
>         at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.performTablesValidation(RDBMSStoreManager.java:3471)
>         at org.datanucleus.store.rdbms.RDBMSStoreManager$ClassAdder.run(RDBMSStoreManager.java:2896)
>         at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:119)
>         at org.datanucleus.store.rdbms.RDBMSStoreManager.manageClasses(RDBMSStoreManager.java:1627)
>         at org.datanucleus.store.rdbms.RDBMSStoreManager.getDatastoreClass(RDBMSStoreManager.java:672)
>         at org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getStatementForCandidates(RDBMSQueryUtils.java:425)
>         at org.datanucleus.store.rdbms.query.JDOQLQuery.compileQueryFull(JDOQLQuery.java:865)
>         at org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:347)
>         at org.datanucleus.store.query.Query.executeQuery(Query.java:1816)
>         at org.datanucleus.store.query.Query.executeWithArray(Query.java:1744)
>         at org.datanucleus.store.query.Query.execute(Query.java:1726)
>         at org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:374)
>         at org.datanucleus.api.jdo.JDOQuery.execute(JDOQuery.java:216)
>         at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.ensureDbInit(MetaStoreDirectSql.java:290)
>         at org.apache.hadoop.hive.metastore.MetaStoreDirectSql.<init>(MetaStoreDirectSql.java:211)
>         at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:274)
>         at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:226)
>         at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
>         at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
>         at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:59)
>         at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:854)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:822)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:816)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:907)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:541)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:80)
>         at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:93)
>         at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:9818)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.callEmbeddedMetastore(HiveMetaStoreClient.java:284)
>         at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:198)
>         at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:118)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>         at org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:84)
>         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:95)
>         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
>         at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
>         at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:5027)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:5095)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:5075)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:5371)
>         at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:323)
>         at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:302)
>         at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:504)
>         at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:405)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:392)
>         at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:363)
>         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.createHiveDB(BaseSemanticAnalyzer.java:264)
>         at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.<init>(BaseSemanticAnalyzer.java:242)
>         at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.<init>(SemanticAnalyzer.java:440)
>         at org.apache.hadoop.hive.ql.QTestUtil.postInit(QTestUtil.java:542)
>         at org.apache.hadoop.hive.cli.control.CliAdapter$1$1.evaluate(CliAdapter.java:88)
>         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>         at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
>         at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
>         at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
>         at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
>         at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
>         at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> Caused by: java.sql.SQLException: You have an error in your SQL syntax; check the manual
that corresponds to your MariaDB server version for the right syntax to use near 'INITIALLY
DEFERRED' at line 1
> Query is: ALTER TABLE KEY_CONSTRAINTS ADD CONSTRAINT KEY_CONSTRAINTS_FK1 FOREIGN KEY
(PARENT_TBL_ID) REFERENCES TBLS (TBL_ID) INITIALLY DEFERRED
> java thread: main
>         at org.mariadb.jdbc.internal.util.LogQueryTool.exceptionWithQuery(LogQueryTool.java:134)
>         at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:184)
>         at org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:329)
>         ... 83 more
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message