spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roch Denis <rde...@exostatic.com>
Subject Re: Help in merging a RDD agaisnt itself using the V of a (K,V).
Date Thu, 24 Jul 2014 01:09:30 GMT
Ah yes, you're quite right with partitions I could probably process a good
chunk of the data but I didn't think a reduce would work? Sorry, I'm still
new to Spark and map reduce in general but I thought that the reduce result
wasn't an RDD and had to fit into memory. If the result of a reduce can be
any size, then yes I can see how to make it work. 

Sorry for not being certain, the doc is not quite clear on that point, at
least to me. 



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Help-in-merging-a-RDD-agaisnt-itself-using-the-V-of-a-K-V-tp10530p10556.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message