spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kevin <kiss.kevin...@gmail.com>
Subject dataframe.foreach VS dataframe.collect().foreach
Date Tue, 26 Jul 2016 07:30:14 GMT
HI ALL:
I don't quite understand the different between : dataframe.foreach and
dataframe.collect().foreach . When to use dataframe.foreach?

I use spark2.0 ,I want to iterate a dataframe to get one colum's value :

this can work out

     blacklistDF.collect().foreach { x =>
        println(s">>>>>>>getString(0)" + x.getAs[String]("uid"))
        val put = new Put(Bytes.toBytes(x.getAs[String]("uid")));
        put.add(Bytes.toBytes("cf"), Bytes.toBytes("uid"),
Bytes.toBytes(x.getAs[String]("uid")))
        hrecords.add(put)
      }

if I use blacklistDF.foreach {....} I will get nothing

Mime
View raw message