spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From thomas lavocat <>
Subject [Spark Streaming MEMORY_ONLY] Understanding Dataflow
Date Wed, 04 Jul 2018 08:26:53 GMT

I have a question on Spark Dataflow. If I understand correctly, all 
received data is sent from the executor to the driver of the application 
prior to task creation.

Then the task embeding the data transit from the driver to the executor 
in order to be processed.

As executor cannot exchange data themselves, in a shuffle, data also 
transit to the driver.

Is that correct ?


To unsubscribe e-mail:

View raw message