spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thierry Herrmann <thierry.herrm...@gmail.com>
Subject Re: How to index each map operation????
Date Wed, 02 Apr 2014 03:10:59 GMT
I'm new to Spark, but isn't this a pure scala question ?

The following seems to work with the spark shell:

$ spark-shell

scala> val rdd = sc.makeRDD(List(10,20,30))
rdd: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[8] at makeRDD at
<console>:12

scala> var cnt = -1
cnt: Int = -1

scala> val rdd2 = rdd.map(i => {cnt+=1;  (cnt,i)} )
rdd2: org.apache.spark.rdd.RDD[(Int, Int)] = MappedRDD[9] at map at
<console>:16

scala> rdd2.collect
res8: Array[(Int, Int)] = Array((0,10), (1,20), (2,30))

Thierry



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-index-each-map-operation-tp3471p3614.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message