spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "N, Bharath" <>
Subject Spark Standalone Authentication and Encryption
Date Wed, 09 Jun 2021 14:10:45 GMT
Hi Team,

We are deploying spark standalone cluster and using features likes rpc authentication with
spark.authenticate.secret and encryption also. We have below queries from our Security teams
on this topic and need your help.

1. How do we make sure  spark.authenticate.secret is not visible to end users as they can
use it to authenticate from other servers as well. Placing in spark-default.conf file allow
all users to know the secret. Even if used  spark.authenticate.secret.file we have to give
read privilege on it which is again a risk.
2. In continuation to above - Can we integrate a vault like hashicorp to store this secret
and use it in runtime ?
3. Encryption when enabled for RPC connections, does it use secret mentioned in spark.authenticate.secret
itself as encryption key ?


NOTICE: All information in and attached to the e-mails below may be proprietary, confidential,
privileged and otherwise protected from improper or erroneous disclosure. If you are not the
sender's intended recipient, you are not authorized to intercept, read, print, retain, copy,
forward, or disseminate this message. If you have erroneously received this communication,
please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all
copies of this message electronic, paper, or otherwise. By transmitting documents via this
email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal
of information via email is voluntary, is offered as a convenience, and is not a secured method
of communication; Not to transmit any payment information E.G. credit card, debit card, checking
account, wire transfer information, passwords, or sensitive and personal information E.G.
Driver's license, DOB, social security, or any other information the user wishes to remain
confidential; To transmit only non-confidential information such as plans, pictures and drawings
and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages
that may arise from the transmittal of documents or including non-confidential information
in the body of an email transmittal. Thank you.

View raw message