spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Austin Nothaft <fnoth...@berkeley.edu>
Subject Re: multiple passes in mapPartitions
Date Wed, 02 Jul 2014 05:44:03 GMT
Hi Zhen,

The Scala iterator trait supports cloning via the duplicate method (http://www.scala-lang.org/api/current/index.html#scala.collection.Iterator@duplicate:(Iterator[A],Iterator[A])).

Regards,

Frank Austin Nothaft
fnothaft@berkeley.edu
fnothaft@eecs.berkeley.edu
202-340-0466

On Jun 13, 2014, at 9:28 PM, zhen <z.he@latrobe.edu.au> wrote:

> Thank you for your suggestion. We will try it out and see how it performs. We
> think the single call to mapPartitions will be faster but we could be wrong.
> It would be nice to have a "clone method" on the iterator.
> 
> 
> 
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/multiple-passes-in-mapPartitions-tp7555p7616.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.


Mime
View raw message