spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From PaweĊ‚ Szulc <paul.sz...@gmail.com>
Subject Re: reverse an rdd
Date Thu, 16 Oct 2014 19:58:51 GMT
Just to have this clear, can you answer with quick yes or no:

Does it mean that when I create RDD from a file and I simply iterate
through it like this:

 sc.textFile("some_text_file.txt").foreach(line => println(line))

then the actual lines might come in different order then they are in the
file?

On Thu, Oct 16, 2014 at 9:13 PM, Sean Owen <sowen@cloudera.com> wrote:

> Since you're concerned with the particular ordering, you will need to
> sort your RDD to ensure the ordering you have in mind. Simply reverse
> the Ordering with Ordering.reverse() and sort by that instead, and
> then use toLocalIterator() I suppose.
>
> Depending on what you're really trying to achieve, there may be a better
> way.
>
> On Thu, Oct 16, 2014 at 2:49 PM, ll <duy.huynh.uiv@gmail.com> wrote:
> > hello... what is the best way to iterate through an rdd backward (last
> > element first, first element last)?  thanks!
> >
> >
> >
> > --
> > View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/reverse-an-rdd-tp16602.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> > For additional commands, e-mail: user-help@spark.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message