hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Gergely (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-19775) Schematool should use HS2 embedded mode in privileged auth mode
Date Fri, 15 Jun 2018 13:49:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16513645#comment-16513645
] 

Miklos Gergely edited comment on HIVE-19775 at 6/15/18 1:48 PM:
----------------------------------------------------------------

HiveSchemaTool is failing in CI with this exception:

{code}
ExecutionFailed: Execution of 'export HIVE_CONF_DIR=/usr/hdp/current/hive-server2/conf/ ;
/usr/hdp/current/hive-server2/bin/schematool -initSchema -dbType hive -metaDbType mysql -userName
hiveuser -passWord [PROTECTED] -verbose' returned 1. ######## Hortonworks #############
This is MOTD message, added for testing in qe infra
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.0.0-1495/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.0.0-1495/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:hive2://?hive.conf.restricted.list=;hive.security.authorization.sqlstd.confwhitelist=*;hive.security.authorization.sqlstd.confwhitelist.append=*;hive.security.authorization.enabled=false;hive.metastore.uris=
Metastore Connection Driver :	 org.apache.hive.jdbc.HiveDriver
Metastore connection User:	 hiveuser
Hive Session ID = 0ade9c28-4437-47fd-a443-cf5535b4ea28
Exception in thread "main" java.lang.RuntimeException: Error applying authorization policy
on hive configuration: Dangling meta character '*' near index 0
*|*
^
	at org.apache.hive.service.cli.CLIService.init(CLIService.java:118)
	at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:62)
	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:205)
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:247)
	at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:86)
	at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:169)
	at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:475)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:581)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: java.util.regex.PatternSyntaxException: Dangling meta character '*' near index
0
*|*
^
	at java.util.regex.Pattern.error(Pattern.java:1957)
	at java.util.regex.Pattern.sequence(Pattern.java:2125)
	at java.util.regex.Pattern.expr(Pattern.java:1998)
	at java.util.regex.Pattern.compile(Pattern.java:1698)
	at java.util.regex.Pattern.<init>(Pattern.java:1351)
	at java.util.regex.Pattern.compile(Pattern.java:1028)
	at org.apache.hadoop.hive.conf.HiveConf.setModifiableWhiteListRegex(HiveConf.java:5612)
	at org.apache.hadoop.hive.ql.security.authorization.plugin.SettableConfigUpdater.setHiveConfWhiteList(SettableConfigUpdater.java:60)
	at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerBase.applyAuthorizationConfigPolicy(RangerHiveAuthorizerBase.java:112)
	at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:946)
	at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:913)
	at org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1882)
	at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:131)
	at org.apache.hive.service.cli.CLIService.init(CLIService.java:115)
	... 17 more

{code}

It seems that it only occurs if Ranger is present.


was (Author: mgergely):
HiveSchemaTool is failing in CI with this exception:

{code}
ExecutionFailed: Execution of 'export HIVE_CONF_DIR=/usr/hdp/current/hive-server2/conf/ ;
/usr/hdp/current/hive-server2/bin/schematool -initSchema -dbType hive -metaDbType mysql -userName
hiveuser -passWord [PROTECTED] -verbose' returned 1. ######## Hortonworks #############
This is MOTD message, added for testing in qe infra
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.0.0-1495/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.0.0-1495/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:hive2://?hive.conf.restricted.list=;hive.security.authorization.sqlstd.confwhitelist=*;hive.security.authorization.sqlstd.confwhitelist.append=*;hive.security.authorization.enabled=false;hive.metastore.uris=
Metastore Connection Driver :	 org.apache.hive.jdbc.HiveDriver
Metastore connection User:	 hiveuser
Hive Session ID = 0ade9c28-4437-47fd-a443-cf5535b4ea28
Exception in thread "main" java.lang.RuntimeException: Error applying authorization policy
on hive configuration: Dangling meta character '*' near index 0
*|*
^
	at org.apache.hive.service.cli.CLIService.init(CLIService.java:118)
	at org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:62)
	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:205)
	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
	at java.sql.DriverManager.getConnection(DriverManager.java:664)
	at java.sql.DriverManager.getConnection(DriverManager.java:247)
	at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:86)
	at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:169)
	at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:475)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:581)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: java.util.regex.PatternSyntaxException: Dangling meta character '*' near index
0
*|*
^
	at java.util.regex.Pattern.error(Pattern.java:1957)
	at java.util.regex.Pattern.sequence(Pattern.java:2125)
	at java.util.regex.Pattern.expr(Pattern.java:1998)
	at java.util.regex.Pattern.compile(Pattern.java:1698)
	at java.util.regex.Pattern.<init>(Pattern.java:1351)
	at java.util.regex.Pattern.compile(Pattern.java:1028)
	at org.apache.hadoop.hive.conf.HiveConf.setModifiableWhiteListRegex(HiveConf.java:5612)
	at org.apache.hadoop.hive.ql.security.authorization.plugin.SettableConfigUpdater.setHiveConfWhiteList(SettableConfigUpdater.java:60)
	at org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerBase.applyAuthorizationConfigPolicy(RangerHiveAuthorizerBase.java:112)
	at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:946)
	at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:913)
	at org.apache.hadoop.hive.ql.session.SessionState.applyAuthorizationPolicy(SessionState.java:1882)
	at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:131)
	at org.apache.hive.service.cli.CLIService.init(CLIService.java:115)
	... 17 more

{code}

> Schematool should use HS2 embedded mode in privileged auth mode
> ---------------------------------------------------------------
>
>                 Key: HIVE-19775
>                 URL: https://issues.apache.org/jira/browse/HIVE-19775
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Thejas M Nair
>            Assignee: Daniel Dai
>            Priority: Critical
>             Fix For: 3.1.0, 4.0.0
>
>         Attachments: HIVE-19775.1.patch, HIVE-19775.2.patch
>
>
> Follow up of  HIVE-19389.
> Authorization checks don't make sense for embedded mode and since it is not used in that
mode it leads to issues if authorization is enabled (eg, username not set).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message