spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Mocanu <amoc...@verticalscope.com>
Subject RE: Splitting RDD and Grouping together to perform computation
Date Fri, 28 Mar 2014 20:54:58 GMT
I think you should sort each RDD

-----Original Message-----
From: yh18190 [mailto:yh18190@gmail.com] 
Sent: March-28-14 4:44 PM
To: user@spark.incubator.apache.org
Subject: Re: Splitting RDD and Grouping together to perform computation

Hi,
Thanks Nanzhu.I tried to implement your suggestion on following scenario.I have RDD of say
24 elements.In that when i partioned into two groups of 12 elements each.Their is loss of
order of elements in partition.Elemest are partitioned randomly.I need to preserve the order
such that the first 12 elements should be 1st partition and 2nd 12 elemts should be in 2nd
partition.
Guys please help me how to main order of original sequence even after partioning....Any solution????
Before Partition:RDD
64
29186
16059
9143
6439
6155
9187
18416
25565
30420
33952
38302
43712
47092
48803
52687
56286
57471
63429
70715
75995
81878
80974
71288
48556
After Partition:In group1 with 12 elements 64, 29186,
18416
30420
33952
38302
43712
47092
56286
81878
80974
71288
48556



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Splitting-RDD-and-Grouping-together-to-perform-computation-tp3153p3447.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message