spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fengdong Yu <fengdo...@everstring.com>
Subject Re: Concurrent execution of actions within a driver
Date Mon, 26 Oct 2015 11:31:42 GMT
not parallel. 

Spark only execute tasks with Action,(‘collect' here)

rdd1.collect  and rdd2.collect are executed sequencely, so Spark execute two tasks one by
one.




> On Oct 26, 2015, at 7:26 PM, praveen S <mylogin13@gmail.com> wrote:
> 
> Does spark run different actions of an rdd within a driver in parallel also?
> 
> Let's say
> class Driver{
> 
> val rdd1= sc. textFile("... ") 
> val rdd2=sc.textFile("....") 
> rdd1. collect //Action 1
> rdd2. collect //Action 2
> 
> }
> 
> Does Spark run Action 1 & 2 run in parallel? ( some kind of a pass through the driver
code and than start the execution)?
> 
> if not than is using threads safe for independent actions/red's?
> 


Mime
View raw message