spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Samel <manojsamelt...@gmail.com>
Subject Re: Do RDD actions run only on driver ?
Date Sun, 19 Jan 2014 18:01:07 GMT
So each action (in driver node) creates a job that can still be executed by
1:N worker node(s) ?


On Sat, Jan 18, 2014 at 10:56 PM, Tathagata Das <tathagata.das1565@gmail.com
> wrote:

> Yes, RDD actions can be called only in the driver program, therefore only
> in the driver node. However, they can be parallelized within the driver
> program by calling multiple actions from multiple threads. The jobs
> corresponding to each action will be executed simultaneously in the Spark
> cluster, sharing the available resources.
>
> TD
>
>
>
>
> On Sat, Jan 18, 2014 at 10:34 PM, Manoj Samel <manojsameltech@gmail.com>wrote:
>
>> Are RDD actions like count etc. run only on driver node or can they be
>> parallelized ?
>>
>> Thanks,
>>
>
>

Mime
View raw message