spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng Lian <lian.cs....@gmail.com>
Subject Re: Spark ThriftServer encounter java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int, auth-conf, auth]
Date Sun, 15 Mar 2015 13:03:34 GMT
Hey Andrew,

Would you please create a JIRA ticket for this? To preserve 
compatibility with existing Hive JDBC/ODBC drivers, Spark SQL's 
HiveThriftServer intercepts some HiveServer2 components and injects 
Spark stuff into it. This makes the implementation details are somewhat 
hacky (e.g. a bunch of reflection tricks were used). We haven't include 
KRB tests in Spark unit/integration test suites, and it's possible that 
HiveThriftServer2 somehow breaks Hive's KRB feature.

Cheng

On 3/14/15 3:43 AM, Andrew Lee wrote:
> When Kerberos is enabled, I get the following exceptions. (Spark 1.2.1 git commit
>
>
>
>
>
>
>
>
> b6eaf77d4332bfb0a698849b1f5f917d20d70e97, Hive 0.13.1, Apache Hadoop 2.4.1) when starting
Spark ThriftServer.
> Command to start thriftserver
> ./start-thriftserver.sh --hiveconf hive.server2.thrift.port=20000 --hiveconf hive.server2.thrift.bind.host=$(hostname)
--master yarn-client
> Error message in spark.log
>
> 2015-03-13 18:26:05,363 ERROR org.apache.hive.service.cli.thrift.ThriftCLIService (ThriftBinaryCLIService.java:run(93))
- Error:
> java.lang.IllegalArgumentException: Unknown auth type: null Allowed values are: [auth-int,
auth-conf, auth]
>          at org.apache.hive.service.auth.SaslQOP.fromString(SaslQOP.java:56)
>          at org.apache.hive.service.auth.HiveAuthFactory.getSaslProperties(HiveAuthFactory.java:118)
>          at org.apache.hive.service.auth.HiveAuthFactory.getAuthTransFactory(HiveAuthFactory.java:133)
>          at org.apache.hive.service.cli.thrift.ThriftBinaryCLIService.run(ThriftBinaryCLIService.java:43)
>          at java.lang.Thread.run(Thread.java:744)
>
> I'm wondering if this is due to the same problem described in HIVE-8154 HIVE-7620 due
to an older code based for the Spark ThriftServer?
> Any insights are appreciated. Currently, I can't get Spark ThriftServer to run against
a Kerberos cluster (Apache 2.4.1).
>
> My hive-site.xml looks like the following for spark/conf.
>
>
>
>
>
>
>
>
> <property>
>    <name>hive.semantic.analyzer.factory.impl</name>
>    <value>org.apache.hcatalog.cli.HCatSemanticAnalyzerFactory</value>
> </property>
> <property>
>    <name>hive.metastore.execute.setugi</name>
>    <value>true</value>
> </property>
> <property>
>    <name>hive.stats.autogather</name>
>    <value>false</value>
> </property>
> <property>
>    <name>hive.session.history.enabled</name>
>    <value>true</value>
> </property>
> <property>
>    <name>hive.querylog.location</name>
>    <value>/home/hive/log/${user.name}</value>
> </property>
> <property>
>    <name>hive.exec.local.scratchdir</name>
>    <value>/tmp/hive/scratch/${user.name}</value>
> </property>
> <property>
>    <name>hive.metastore.uris</name>
>    <value>thrift://somehostname:9083</value>
> </property>
> <!-- HIVE SERVER 2 -->
> <property>
>    <name>hive.server2.authentication</name>
>    <value>KERBEROS</value>
> </property>
> <property>
>    <name>hive.server2.authentication.kerberos.principal</name>
>    <value>***</value>
> </property>
> <property>
>    <name>hive.server2.authentication.kerberos.keytab</name>
>    <value>***</value>
> </property>
> <property>
>    <name>hive.server2.thrift.sasl.qop</name>
>    <value>auth</value>
>    <description>Sasl QOP value; one of 'auth', 'auth-int' and 'auth-conf'</description>
> </property>
> <property>
>    <name>hive.server2.enable.impersonation</name>
>    <description>Enable user impersonation for HiveServer2</description>
>    <value>true</value>
> </property>
> <!-- HIVE METASTORE -->
> <property>
>    <name>hive.metastore.sasl.enabled</name>
>    <value>true</value>
> </property>
> <property>
>    <name>hive.metastore.kerberos.keytab.file</name>
>    <value>***</value>
> </property>
> <property>
>    <name>hive.metastore.kerberos.principal</name>
>    <value>***</value>
> </property>
> <property>
>    <name>hive.metastore.cache.pinobjtypes</name>
>    <value>Table,Database,Type,FieldSchema,Order</value>
> </property>
> <property>
>    <name>hdfs_sentinel_file</name>
>    <value>***</value>
> </property>
> <property>
>    <name>hive.metastore.warehouse.dir</name>
>    <value>/hive</value>
> </property>
> <property>
>    <name>hive.metastore.client.socket.timeout</name>
>    <value>600</value>
> </property>
> <property>
>    <name>hive.warehouse.subdir.inherit.perms</name>
>    <value>true</value>
> </property> 		 	   		


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message