spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: Pairwise Processing of a List
Date Mon, 26 Jan 2015 01:33:30 GMT
Sean,

On Mon, Jan 26, 2015 at 10:28 AM, Sean Owen <sowen@cloudera.com> wrote:

> Note that RDDs don't really guarantee anything about ordering though,
> so this only makes sense if you've already sorted some upstream RDD by
> a timestamp or sequence number.
>

Speaking of order, is there some reading on guarantees and non-guarantees
about order in RDDs? For example, when reading a file and doing
zipWithIndex, can I assume that the lines are numbered in order? Does this
hold for receiving data from Kafka, too?

Tobias

Mime
View raw message