spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Litvak <boris.lit...@skf.com>
Subject [Spark RDD] Persisting Spark RDDs across spark contexts/applications - options
Date Thu, 04 Jun 2020 07:11:11 GMT
I would like to cache Apache Spark RDDs and share them between Spark applications.

Alluxio (Tachyon), Redis & Ignite all offer such capabilities.

For instance, see Ignite's proposal:
[cid:image003.png@01D63A58.74971600]

Are there any comparison studies on performance/maintenance burden/installation experience
of the above frameworks?

If you have you had any experience using spark with any of these technologies, please share.

Thanks, Boris

Mime
View raw message