spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rapelly kartheek <kartheek.m...@gmail.com>
Subject Fwd: RDDs
Date Thu, 04 Sep 2014 06:24:22 GMT
---------- Forwarded message ----------
From: rapelly kartheek <kartheek.mbms@gmail.com>
Date: Thu, Sep 4, 2014 at 11:49 AM
Subject: Re: RDDs
To: "Liu, Raymond" <raymond.liu@intel.com>


Thank you Raymond.
I am more clear now. So, if an rdd is replicated over multiple nodes (i.e.
say two sets of nodes as it is a collection of chunks), can we run two jobs
concurrently and seperately on these two sets of nodes?


On Thu, Sep 4, 2014 at 11:38 AM, Liu, Raymond <raymond.liu@intel.com> wrote:

> Actually, a replicated RDD and a parallel job on the same RDD, this two
> conception is not related at all.
> A replicated RDD just store data on multiple node, it helps with HA and
> provide better chance for data locality. It is still one RDD, not two
> separate RDD.
> While regarding run two jobs on the same RDD, it doesn't matter that the
> RDD is replicated or not. You can always do it if you wish to.
>
>
> Best Regards,
> Raymond Liu
>
> -----Original Message-----
> From: Kartheek.R [mailto:kartheek.mbms@gmail.com]
> Sent: Thursday, September 04, 2014 1:24 PM
> To: user@spark.incubator.apache.org
> Subject: RE: RDDs
>
> Thank you Raymond and Tobias.
> Yeah, I am very clear about what I was asking. I was talking about
> "replicated" rdd only. Now that I've got my understanding about job and
> application validated, I wanted to know if we can replicate an rdd and run
> two jobs (that need same rdd) of an application in parallel?.
>
> -Karthk
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/RDDs-tp13343p13416.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional
> commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message