spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Iulian DragoČ™ <iulian.dra...@typesafe.com>
Subject Re: Accessing Kerberos Secured HDFS Resources from Spark on Mesos
Date Sun, 28 Jun 2015 12:12:55 GMT
This is something we (at Typesafe) also thought about, but didn't start
yet. It would be good to pool efforts.

On Sat, Jun 27, 2015 at 12:44 AM, Dave Ariens <dariens@blackberry.com>
wrote:

>  Fair. I will look into an alternative with a generated delegation token.
>   However the same issue exists.   How can I have the executor run some
> arbitrary code when it gets a task assignment and before it proceeds to
> process it's resources?
>
>     *From: *Marcelo Vanzin
> *Sent: *Friday, June 26, 2015 6:20 PM
> *To: *Dave Ariens
> *Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org
> *Subject: *Re: Accessing Kerberos Secured HDFS Resources from Spark on
> Mesos
>
>   On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens <dariens@blackberry.com>
> wrote:
>
>>  Would there be any way to have the task instances in the slaves call
>> the UGI login with a principal/keytab provided to the driver?
>>
>
>  That would only work with a very small number of executors. If you have
> many login requests in a short period of time with the same principal, the
> KDC will start to deny logins. That's why delegation tokens are used
> instead of explicit logins.
>
>  --
> Marcelo
>



-- 

--
Iulian Dragos

------
Reactive Apps on the JVM
www.typesafe.com

Mime
View raw message