spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Evans <>
Subject What's the deal with --proxy-user?
Date Wed, 06 Nov 2019 22:49:30 GMT
Hi all,

I'm trying to understand if the --proxy-user parameter to spark-submit is
deprecated, or something similar?  The reason I ask is because it's hard to
find documentation really talking about it.  The Spark Security doc
<> doesn't mention it at
all, nor does the YARN doc
<>.  It does show
up in the source code in this README file
Is that supposed to be the definitive documentation for the option?

Looking into vendor specific docs, it disappears between CDH 5.9
and CDH 5.10
docs.  I couldn't find it anywhere in the HDP documentation, but I'm not as
familiar with navigating there.

And perhaps I'm reading a bit too much into the tea leaves here, but the
comments on SPARK-13478
<> (which
removed the ability to specify --proxy-user along with --principal and
--keytab) seem to read like "you shouldn't be using it; you should just use
principal and keytab".

So, am I going crazy or is this really not clearly documented?  Is it a
deprecated feature or will it continue to be supported long term?  Any kind
of insight is appreciated.

View raw message