spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Or <and...@databricks.com>
Subject Re: How can I start history-server with kerberos HDFS ?
Date Wed, 03 Sep 2014 23:52:50 GMT
Hi Zhanfeng,

You will need to set these through SPARK_HISTORY_OPTS in conf/spark-env.sh.
This is documented here: http://spark.apache.org/docs/latest/monitoring.html
.

Let me know if you have it working,
-Andrew



2014-09-03 11:14 GMT-07:00 Marcelo Vanzin <vanzin@cloudera.com>:

> The history server (and other Spark daemons) do not read
> spark-defaults.conf. There's a bug open to implement that
> (SPARK-2098), and an open PR to fix it, but it's still not in Spark.
>
> On Wed, Sep 3, 2014 at 11:00 AM, Zhanfeng Huo <huozhanfeng@gmail.com>
> wrote:
> > Hi,
> >
> >     I have seted properties in conf/spark-defaults.conf  and start with
> > command "./sbin/start-history-server.sh /tmp/spark-events". It get errors
> > and seems that the properties in spark-defaults.conf  file doesn't
> effect.
> > How can I solve this problem(Enable properties in spark-defaults.conf
> when
> > start spark history-server) ?
> >
> >    14/09/04 01:44:04 WARN NativeCodeLoader: Unable to load native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > 14/09/04 01:44:05 WARN UserGroupInformation: PriviledgedActionException
> > as:root (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS
> > initiate failed [Caused by GSSException: No valid credentials provided
> > (Mechanism level: Failed to find any Kerberos tgt)]
> > 14/09/04 01:44:05 WARN Client: Exception encountered while connecting to
> the
> > server : javax.security.sasl.SaslException: GSS initiate failed [Caused
> by
> > GSSException: No valid credentials provided (Mechanism level: Failed to
> find
> > any Kerberos tgt)]
> > 14/09/04 01:44:05 WARN UserGroupInformation: PriviledgedActionException
> > as:root (auth:KERBEROS) cause:java.io.IOException:
> > javax.security.sasl.SaslException: GSS initiate failed [Caused by
> > GSSException: No valid credentials provided (Mechanism level: Failed to
> find
> > any Kerberos tgt)]
> > Exception in thread "main" java.io.IOException: Failed on local
> exception:
> > java.io.IOException: javax.security.sasl.SaslException: GSS initiate
> failed
> > [Caused by GSSException: No valid credentials provided (Mechanism level:
> > Failed to find any Kerberos tgt)]; Host Details :
> >
> >     #history-server
> >     spark.history.kerberos.enabled true
> >     park.history.kerberos.principal test/spark@test
> >     spark.history.kerberos.keytab /home/test/test_spark.keytab
> >     spark.eventLog.enabled true
> >
> > ________________________________
> > Zhanfeng Huo
>
>
>
> --
> Marcelo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message