spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@cs.berkeley.edu>
Subject Re: Getting the partition position of cached RDD?
Date Mon, 02 Sep 2013 07:28:03 GMT
Does this help you? https://github.com/mesos/spark/pull/832


--
Reynold Xin, AMPLab, UC Berkeley
http://rxin.org



On Mon, Sep 2, 2013 at 3:24 PM, Wenlei Xie <wenlei.xie@gmail.com> wrote:

> Hi,
>
> I am wondering if it is possible to get the partition position of cached
> RDD? I am asking this because I am trying to avoid shuffling when
> performing coalesce operation. And the size of my partitions could be quite
> imbalance thus CoalescedRDD would probably not be a good solution in my
> case.
>
> Thank you!
>
> Best,
> Wenlei
>
> --
> Wenlei Xie (谢文磊)
>
> Department of Computer Science
> 5132 Upson Hall, Cornell University
> Ithaca, NY 14853, USA
> Phone: (607) 255-5577
> Email: wenlei.xie@gmail.com
>

Mime
View raw message