spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Miller <cmiller11...@gmail.com>
Subject Re: Does parallelize and collect preserve the original order of list?
Date Wed, 16 Mar 2016 07:47:33 GMT
Short answer: Nope

Less short answer: Spark is not designed to maintain sort order in this
case... it *may*, but there's no guarantee... generally, it would not be in
the same order unless you implement something to order by and then sort the
result based on that.

--
Chris Miller

On Wed, Mar 16, 2016 at 10:16 AM, JoneZhang <joyoungzhang@gmail.com> wrote:

> Step1
>             List<String> items = new ArrayList<String>();items.addAll(XXX);
>             javaSparkContext.parallelize(items).saveAsTextFile(output);
> Step2
>             final List<String> items2 = ctx.textFile(output).collect();
>
> Does items and items2 has the same order?
>
>
> Besh wishes.
> Thanks.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Does-parallelize-and-collect-preserve-the-original-order-of-list-tp26512.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message