spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: Semantics of foreachPartition()
Date Thu, 18 Dec 2014 09:51:02 GMT
Hi again,

On Thu, Dec 18, 2014 at 6:43 PM, Tobias Pfeiffer <tgp@preferred.jp> wrote:
>
>         tmpRdd.foreachPartition(iter => {
>           iter.map(item => {
>             println("xyz: " + item)
>           })
>         })
>

Uh, with iter.foreach(...) it works... the reason being apparently that
iter.map() returns itself an iterator, is thus evaluated lazily (in this
case: never), while iter.foreach() is evaluated immediately.

Thanks
Tobias

Mime
View raw message