spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sudarshan Rangarajan <tr.sudars...@gmail.com>
Subject User impersonation with Kerberos and Delegation tokens
Date Thu, 26 May 2016 00:49:15 GMT
Hi there,

I'm using SparkLauncher API from Spark v1.6.1, to submit a Spark job to
YARN. The service from where this API will be invoked will need to talk to
other services on our cluster via Kerberos (ex. HDFS, YARN etc.). Also, my
service expects to impersonate its then logged-in user during job
submission to YARN.

Can someone comment on/confirm, the following two observations ?

1. From this Github PR
<https://github.com/apache/spark/pull/11358/commits/0159499a55591f25c690bfdfeecfa406142be02b>,
it seems that SparkSubmit doesn't support user impersonation when Kerberos
credentials of the real user are specified.

2. SPARK-14743 <https://issues.apache.org/jira/browse/SPARK-14743> - It
seems that Spark doesn't yet support acquisition of delegation tokens
during job submission to YARN irrespective of user impersonation. This Github
discussion <https://github.com/apache/spark/pull/11358> is also relevant to
this scenario.

Thanks,
Sudarshan

Mime
View raw message