spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhanfeng Huo" <huozhanf...@gmail.com>
Subject Re: Re: How can I start history-server with kerberos HDFS ?
Date Thu, 04 Sep 2014 04:57:19 GMT
Thanks for your help.

It works after setting SPARK_HISTORY_OPTS.



Zhanfeng Huo
 
From: Andrew Or
Date: 2014-09-04 07:52
To: Marcelo Vanzin
CC: Zhanfeng Huo; user
Subject: Re: How can I start history-server with kerberos HDFS ?
Hi Zhanfeng,

You will need to set these through SPARK_HISTORY_OPTS in conf/spark-env.sh. This is documented
here: http://spark.apache.org/docs/latest/monitoring.html.

Let me know if you have it working,
-Andrew



2014-09-03 11:14 GMT-07:00 Marcelo Vanzin <vanzin@cloudera.com>:
The history server (and other Spark daemons) do not read
spark-defaults.conf. There's a bug open to implement that
(SPARK-2098), and an open PR to fix it, but it's still not in Spark.

On Wed, Sep 3, 2014 at 11:00 AM, Zhanfeng Huo <huozhanfeng@gmail.com> wrote:
> Hi,
>
>     I have seted properties in conf/spark-defaults.conf  and start with
> command "./sbin/start-history-server.sh /tmp/spark-events". It get errors
> and seems that the properties in spark-defaults.conf  file doesn't effect.
> How can I solve this problem(Enable properties in spark-defaults.conf when
> start spark history-server) ?
>
>    14/09/04 01:44:04 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/09/04 01:44:05 WARN UserGroupInformation: PriviledgedActionException
> as:root (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS
> initiate failed [Caused by GSSException: No valid credentials provided
> (Mechanism level: Failed to find any Kerberos tgt)]
> 14/09/04 01:44:05 WARN Client: Exception encountered while connecting to the
> server : javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> 14/09/04 01:44:05 WARN UserGroupInformation: PriviledgedActionException
> as:root (auth:KERBEROS) cause:java.io.IOException:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> Exception in thread "main" java.io.IOException: Failed on local exception:
> java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
> [Caused by GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)]; Host Details :
>
>     #history-server
>     spark.history.kerberos.enabled true
>     park.history.kerberos.principal test/spark@test
>     spark.history.kerberos.keytab /home/test/test_spark.keytab
>     spark.eventLog.enabled true
>
> ________________________________
> Zhanfeng Huo



--
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message