spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhishek R. Singh" <abhis...@tetrationanalytics.com>
Subject Re: Transformation not happening for reduceByKey or GroupByKey
Date Fri, 21 Aug 2015 13:17:39 GMT
You had:

> RDD.reduceByKey((x,y) => x+y)
> RDD.take(3)

Maybe try:

> rdd2 = RDD.reduceByKey((x,y) => x+y)
> rdd2.take(3)

-Abhishek-

On Aug 20, 2015, at 3:05 AM, satish chandra j <jsatishchandra@gmail.com> wrote:

> HI All,
> I have data in RDD as mentioned below:
> 
> RDD : Array[(Int),(Int)] = Array((0,1), (0,2),(1,20),(1,30),(2,40))
> 
> 
> I am expecting output as Array((0,3),(1,50),(2,40)) just a sum function on Values for
each key
> 
> Code:
> RDD.reduceByKey((x,y) => x+y)
> RDD.take(3)
> 
> Result in console:
> RDD: org.apache.spark.rdd.RDD[(Int,Int)]= ShuffledRDD[1] at reduceByKey at <console>:73
> res:Array[(Int,Int)] = Array()
> 
> Command as mentioned
> 
> dse spark --master local --jars postgresql-9.4-1201.jar -i  <ScriptFile>
> 
> 
> Please let me know what is missing in my code, as my resultant Array is empty
> 
> 
> 
> Regards,
> Satish
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message