spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kevin <kiss.kevin...@gmail.com>
Subject Re: dataframe.foreach VS dataframe.collect().foreach
Date Tue, 26 Jul 2016 07:53:04 GMT
thank you Chanh

2016-07-26 15:34 GMT+08:00 Chanh Le <giaosudau@gmail.com>:

> Hi Ken,
>
> *blacklistDF -> just DataFrame *
> Spark is lazy until you call something like* collect, take, write* it
> will execute the hold process *like you do map or filter before you
> collect*.
> That mean until you call collect spark* do nothing* so you df would not
> have any data -> can’t call foreach.
> Call collect execute the process -> get data -> foreach is ok.
>
>
> On Jul 26, 2016, at 2:30 PM, kevin <kiss.kevin119@gmail.com> wrote:
>
>  blacklistDF.collect()
>
>
>

Mime
View raw message