spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Davidson (JIRA)" <>
Subject [jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module
Date Sun, 22 Mar 2015 22:22:11 GMT


Aaron Davidson commented on SPARK-6229:

Two points:

1. network/common as written is intended to be independent of Spark; it's just a small, low-level
transport layer which was intended to be easy to make stable and fast, while exposing a much
narrower API than Netty. In this case, it *is* useful to have separation of concerns between
the low-level transport and the application-level concerns like encryption or authentication,
which depend at least partially on Spark's semantics (i.e., SecurityManager).

2. Even if this were not the case, it is useful to expose minimal APIs between components,
for the sake of unit testing and clearly demarcating different components. For this alone
I would favor having an EncryptionHandler of some sort rather than an "if (sasl) { pipeline.addLast(new
SaslHandler()) }". Someone reading the code has to only understand what's needed by the high-level
"encryption" API rather than needing to know what SASL is or even having the option of reaching
into the SASL internals. The separation is clearer.

These are why I would prefer to have a layer of indirection (a factory, some might say) between
SASL or SSL and the TransportContext itself. I want to keep the TransportContext simple and
easy to verify, without embedded notions of either SASL or SSL.

> Support SASL encryption in network/common module
> ------------------------------------------------
>                 Key: SPARK-6229
>                 URL:
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core
>            Reporter: Marcelo Vanzin
> After SASL support has been added to network/common, supporting encryption should be
rather simple. Encryption is supported for DIGEST-MD5 and GSSAPI. Since the latter requires
a valid kerberos login to work (and so doesn't really work with executors), encryption would
require the use of DIGEST-MD5.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message