spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <>
Subject Re: random shuffle streaming RDDs?
Date Mon, 03 Nov 2014 16:48:05 GMT
I think the answer will be the same in streaming as in the core. You
want a random permutation of an RDD? in general RDDs don't have
ordering at all -- excepting when you sort for example -- so a
permutation doesn't make sense. Do you just want a well-defined but
random ordering of the data? Do you just want to (re-)assign elements
randomly to partitions?

On Mon, Nov 3, 2014 at 4:33 PM, Josh J <> wrote:
> Hi,
> Is there a nice or optimal method to randomly shuffle spark streaming RDDs?
> Thanks,
> Josh

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message