spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <>
Subject Re: Encryption on RDDs or in-memory/cache on Apache Spark
Date Sun, 02 Aug 2015 09:54:22 GMT
I think you use case can already be implemented with HDFS encryption and/or
SealedObject, if you look for sth like Altibase.

If you create a JIRA you may want to set the bar a little bit higher and
propose sth like MIT cryptdb:

Le ven. 31 juil. 2015 à 10:17, Matthew O'Reilly <> a
écrit :

> Hi,
> I am currently working on the latest version of Apache Spark (1.4.1),
> pre-built package for Hadoop 2.6+.
> Is there any feature in Spark/Hadoop to encrypt RDDs or in-memory/cache
> (something similar is Altibase's HDB:
> when running applications in Spark? Or is there an external
> library/framework which could be used to encrypt RDDs or in-memory/cache in
> Spark?
> I discovered it is possible to encrypt the data, and encapsulate it into
> RDD. However, I feel this affects Spark's fast data processing as it is
> slower to encrypt the data, and then encapsulate it to RDD; it's then a two
> step process. Encryption and storing data should be done parallel.
> Any help would be appreciated.
> Many thanks,
> Matthew
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message