I agree with Wenchen that we'd remove the prefix when passing to a source, so you could use the same "spark.yarn.keytab" option in both places. But I think the problem is that "spark.yarn.keytab" still needs to be set, and it clearly isn't in a shared namespace for catalog options. So I think we would still need a solution for existing options. I'm more comfortable with a white list for existing options that we want to maintain compatibility with.

rb



On Mon, Sep 24, 2018 at 11:52 PM tigerquoll <tigerquoll@outlook.com> wrote:
To give some Kerberos specific examples, The spark-submit args:
-–conf spark.yarn.keytab=path_to_keytab -–conf
spark.yarn.principal=principal@REALM.COM

are currently not passed through to the data sources.





--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org



--
Ryan Blue
Software Engineer
Netflix