spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew O'Reilly <>
Subject Encryption on RDDs or in-memory/cache on Apache Spark
Date Fri, 31 Jul 2015 08:17:27 GMT

I am currently working on the latest version of Apache Spark (1.4.1), pre-built package for
Hadoop 2.6+.

Is there any feature in Spark/Hadoop to encrypt RDDs or in-memory/cache (something similar
is Altibase's HDB:
when running applications in Spark? Or is there an external library/framework which could
be used to encrypt RDDs or in-memory/cache in Spark?

I discovered it is possible to encrypt the data, and encapsulate it into RDD. However, I feel
this affects Spark's fast data processing as it is slower to encrypt the data, and then encapsulate
it to RDD; it's then a two step process. Encryption and storing data should be done parallel.

Any help would be appreciated.

Many thanks,

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message