spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Evans <jeffrey.wayne.ev...@gmail.com>
Subject What's the deal with --proxy-user?
Date Wed, 06 Nov 2019 22:49:30 GMT
Hi all,

I'm trying to understand if the --proxy-user parameter to spark-submit is
deprecated, or something similar?  The reason I ask is because it's hard to
find documentation really talking about it.  The Spark Security doc
<https://spark.apache.org/docs/latest/security.html> doesn't mention it at
all, nor does the YARN doc
<https://spark.apache.org/docs/latest/running-on-yarn.html>.  It does show
up in the source code in this README file
<https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/security/README.md>.
Is that supposed to be the definitive documentation for the option?

Looking into vendor specific docs, it disappears between CDH 5.9
<https://docs.cloudera.com/documentation/enterprise/5-9-x/topics/sg_spark_auth.html>
and CDH 5.10
<https://docs.cloudera.com/documentation/enterprise/5-10-x/topics/sg_spark_auth.html>
docs.  I couldn't find it anywhere in the HDP documentation, but I'm not as
familiar with navigating there.

And perhaps I'm reading a bit too much into the tea leaves here, but the
comments on SPARK-13478
<https://issues.apache.org/jira/browse/SPARK-13478> (which
removed the ability to specify --proxy-user along with --principal and
--keytab) seem to read like "you shouldn't be using it; you should just use
principal and keytab".

So, am I going crazy or is this really not clearly documented?  Is it a
deprecated feature or will it continue to be supported long term?  Any kind
of insight is appreciated.

Mime
View raw message