spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chanh Le <giaosu...@gmail.com>
Subject Re: dataframe.foreach VS dataframe.collect().foreach
Date Tue, 26 Jul 2016 07:34:57 GMT
Hi Ken,

blacklistDF -> just DataFrame 
Spark is lazy until you call something like collect, take, write it will execute the hold
process like you do map or filter before you collect.
That mean until you call collect spark do nothing so you df would not have any data ->
can’t call foreach.
Call collect execute the process -> get data -> foreach is ok.


> On Jul 26, 2016, at 2:30 PM, kevin <kiss.kevin119@gmail.com> wrote:
> 
>  blacklistDF.collect()


Mime
View raw message