spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dipl.-Inf. Rico Bergmann" <>
Subject Spark DataSets and multiple write(.) calls
Date Mon, 19 Nov 2018 08:03:32 GMT

I have a SparkSQL programm, having one input and 6 ouputs (write). When
executing this programm every call to write(.) executes the plan. My
problem is, that I want all these writes to happen in parallel (inside
one execution plan), because all writes have a common and compute
intensive subpart, that can be shared by all plans. Is there a
possibility to do this? (Caching is not a solution because the input
dataset is way to large...)

Hoping for advises ...

Best, Rico B.

Diese E-Mail wurde von Avast Antivirus-Software auf Viren gepr├╝ft.

To unsubscribe e-mail:

View raw message