spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: whether a certain piece can be assigned to a specicified node by some codes in my program.
Date Sun, 27 Mar 2016 13:30:28 GMT
Please take a look at the MyRDD class in:

core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala

There is scaladoc for the class. See how getPreferredLocations() is
implemented.

Cheers

On Sun, Mar 27, 2016 at 2:01 AM, chenyong <cy163@hotmail.com> wrote:

> Thank you Ted for your reply.
>
> Your example code is little bit difficult for me to understanding. Could
> youn please give me an explaination about it.
>
>
> ------------------------------
> From: cy163@hotmail.com
> To: user@spark.apache.org
> Subject: whether a certain piece can be assigned to a specicified node by
> some codes in my program.
> Date: Sat, 26 Mar 2016 22:41:49 +0800
>
>
> I am newbie to the greate spark framework. After reading some meterials
> about spark, I know that a RDD dataset are actually  broken into pieces and
> distributed among serveral nodes. I am wondering whether a certain piece
> can be assigned to a specicified node by some codes in my program. Or
> pieces are  randomly assigned to nodes.
>
> Thanks in advance
>
> Felix Chen
>

Mime
View raw message