spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abarah <selme...@yahoo.com>
Subject Using reference for RDD is safe?
Date Wed, 15 Jul 2015 03:30:08 GMT
Hello, I am wondering what will happen if I use a reference for transforming
rdd, for example:

def func1(rdd: RDD[Int]): RDD[Int] = {
    rdd.map(x => x * 2) // example transformation, but I am using a more
complex function
}

def main() {
   .....
   val myrdd = sc.parallelize(1 to 1000000)
   val myrdd2 = func1(myrdd)
   myrdd2.count()
}

The above is just an example. I am wondering if it is safe to use reference
like this. Thank you.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Using-reference-for-RDD-is-safe-tp23843.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message