hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongzhi Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20016) Investigate random test failure
Date Thu, 28 Jun 2018 12:21:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16526257#comment-16526257
] 

Yongzhi Chen commented on HIVE-20016:
-------------------------------------

The test failed for the second time. Revert the test from master for now.
The thread (2128)who is working on the failed stack worked on Session e14a5024-5241-4ae6-84f5-b5e1cbb6fa15.
Around that time, HiveServer2-Background-Pool: Thread-2154 works on that session two (because
of Async compilation):
{noformat}
2018-06-27T22:24:46,717  INFO [HiveServer2-Background-Pool: Thread-2154] sqlstd.SQLStdHiveAccessController:
Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=e14a5024-5241-4ae6-84f5-b5e1cbb6fa15,
clientType=HIVESERVER2]

....
2018-06-27T22:24:46,719 DEBUG [HiveServer2-Background-Pool: Thread-2154] session.SessionState:
Session is using authorization class class org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl
{noformat}

GetSchemasOperation.runInternal can not blocked by Compile lock for the session, so it can
be called  when HiveServer2-Background-Pool: Thread-2154 is compiling. And the getAuthorizationMode()
and setupAuth() are never synchronized, so there is race condition.
{noformat}
2018-06-27T22:24:46,720  WARN [HiveServer2-Handler-Pool: Thread-2128] thrift.ThriftCLIService:
Error getting schemas: 
java.lang.RuntimeException: java.lang.AssertionError: Authorization plugins not initialized!
	at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) ~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_102]
	at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_102]
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682) ~[hadoop-common-3.1.0.jar:?]
	at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at com.sun.proxy.$Proxy41.getSchemas(Unknown Source) ~[?:?]
	at org.apache.hive.service.cli.CLIService.getSchemas(CLIService.java:349) ~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.cli.thrift.ThriftCLIService.GetSchemas(ThriftCLIService.java:610)
[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1617)
[hive-service-rpc-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$GetSchemas.getResult(TCLIService.java:1602)
[hive-service-rpc-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) [libthrift-0.9.3.jar:0.9.3]
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) [libthrift-0.9.3.jar:0.9.3]
	at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
[libthrift-0.9.3.jar:0.9.3]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_102]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_102]
	at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
Caused by: java.lang.AssertionError: Authorization plugins not initialized!
	at org.apache.hadoop.hive.ql.session.SessionState.getAuthorizationMode(SessionState.java:1846)
~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hadoop.hive.ql.session.SessionState.isAuthorizationModeV2(SessionState.java:1850)
~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.cli.operation.MetadataOperation.isAuthV2Enabled(MetadataOperation.java:126)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.cli.operation.GetSchemasOperation.runInternal(GetSchemasOperation.java:68)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.cli.operation.Operation.run(Operation.java:251) ~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at org.apache.hive.service.cli.session.HiveSessionImpl.getSchemas(HiveSessionImpl.java:625)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_102]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_102]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_102]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_102]
	at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
~[hive-service-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
	... 18 more

{noformat}



> Investigate random test failure 
> --------------------------------
>
>                 Key: HIVE-20016
>                 URL: https://issues.apache.org/jira/browse/HIVE-20016
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 4.0.0
>            Reporter: Yongzhi Chen
>            Assignee: Yongzhi Chen
>            Priority: Major
>
> org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3 failed with:
> java.lang.AssertionError: Concurrent Statement failed: org.apache.hive.service.cli.HiveSQLException:
java.lang.AssertionError: Authorization plugins not initialized!
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.apache.hive.jdbc.TestJdbcWithMiniHS2.finishTasks(TestJdbcWithMiniHS2.java:374)
> 	at org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3(TestJdbcWithMiniHS2.java:304)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message