spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Herman van Hövell tot Westerflier <hvanhov...@questtec.nl>
Subject Re: Where is DataFrame.scala in 2.0?
Date Fri, 03 Jun 2016 15:04:55 GMT
Hi Gerhard,

DataFrame and DataSet have been merged in Spark 2.0. A DataFrame is now a
DataSet that contains Row objects. We still maintain a type alias for
DataFrame:
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/package.scala#L45

HTH

Kind regards,

Herman van Hövell tot Westerflier

2016-06-03 17:01 GMT+02:00 Gerhard Fiedler <gfiedler@algebraixdata.com>:

> When I look at the sources in Github, I see DataFrame.scala at
> https://github.com/apache/spark/blob/branch-1.6/sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala
> in the 1.6 branch. But when I change the branch to branch-2.0 or master, I
> get a 404 error. I also can’t find the file in the directory listings, for
> example
> https://github.com/apache/spark/tree/branch-2.0/sql/core/src/main/scala/org/apache/spark/sql
> (for branch-2.0).
>
>
>
> It seems that quite a few APIs use the DataFrame class, even in 2.0. Can
> someone please point me to its location, or otherwise explain why it is not
> there?
>
>
>
> Thanks,
>
> Gerhard
>
>
>

Mime
View raw message