spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dave Ariens <dari...@blackberry.com>
Subject Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
Date Fri, 26 Jun 2015 22:44:27 GMT
Fair. I will look into an alternative with a generated delegation token.   However the same
issue exists.   How can I have the executor run some arbitrary code when it gets a task assignment
and before it proceeds to process it's resources?

From: Marcelo Vanzin
Sent: Friday, June 26, 2015 6:20 PM
To: Dave Ariens
Cc: Tim Chen; Olivier Girardot; user@spark.apache.org
Subject: Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos


On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens <dariens@blackberry.com<mailto:dariens@blackberry.com>>
wrote:
Would there be any way to have the task instances in the slaves call the UGI login with a
principal/keytab provided to the driver?

That would only work with a very small number of executors. If you have many login requests
in a short period of time with the same principal, the KDC will start to deny logins. That's
why delegation tokens are used instead of explicit logins.

--
Marcelo

Mime
View raw message