spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wenlei Xie <wenlei....@gmail.com>
Subject Getting the partition position of cached RDD?
Date Mon, 02 Sep 2013 07:24:15 GMT
Hi,

I am wondering if it is possible to get the partition position of cached
RDD? I am asking this because I am trying to avoid shuffling when
performing coalesce operation. And the size of my partitions could be quite
imbalance thus CoalescedRDD would probably not be a good solution in my
case.

Thank you!

Best,
Wenlei

-- 
Wenlei Xie (谢文磊)

Department of Computer Science
5132 Upson Hall, Cornell University
Ithaca, NY 14853, USA
Phone: (607) 255-5577
Email: wenlei.xie@gmail.com

Mime
View raw message