spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: random shuffle streaming RDDs?
Date Mon, 03 Nov 2014 16:48:05 GMT
I think the answer will be the same in streaming as in the core. You
want a random permutation of an RDD? in general RDDs don't have
ordering at all -- excepting when you sort for example -- so a
permutation doesn't make sense. Do you just want a well-defined but
random ordering of the data? Do you just want to (re-)assign elements
randomly to partitions?

On Mon, Nov 3, 2014 at 4:33 PM, Josh J <joshjdevl@gmail.com> wrote:
> Hi,
>
> Is there a nice or optimal method to randomly shuffle spark streaming RDDs?
>
> Thanks,
> Josh

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message