import org.apache.spark.sql.catalyst.encoders.RowEncoderimplicit val encoder = RowEncoder(df.schema)df.mapPartitions(_.take(1))On Aug 3, 2016, at 04:55, Dragisa Krsmanovic <firstname.lastname@example.org> wrote:I am trying to use mapPartitions on DataFrame.
val df: DataFrame = Seq((1,"one"), (2, "two")).toDF("id", "name")
df.mapPartitions(_.take(1))I am getting:
Unable to find encoder for type stored in a Dataset. Primitive types (Int, String, etc) and Product types (case classes) are supported by importing spark.implicits._ Support for serializing other types will be added in future releases.Since DataFrame is Dataset[Row], I was expecting encoder for Row to be there.What's wrong with my code ?