spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dipl.-Inf. Rico Bergmann" <i...@ricobergmann.de>
Subject Spark DataSets and multiple write(.) calls
Date Mon, 19 Nov 2018 08:03:32 GMT
Hi!

I have a SparkSQL programm, having one input and 6 ouputs (write). When
executing this programm every call to write(.) executes the plan. My
problem is, that I want all these writes to happen in parallel (inside
one execution plan), because all writes have a common and compute
intensive subpart, that can be shared by all plans. Is there a
possibility to do this? (Caching is not a solution because the input
dataset is way to large...)

Hoping for advises ...

Best, Rico B.


---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr├╝ft.
https://www.avast.com/antivirus


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message