spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <>
Subject [SQL] Why does (0 to 9).toDF("num").as[String] work?
Date Sat, 13 Aug 2016 20:17:54 GMT

Just ran into it and can't explain why it works. Please help me understand it.

Q1: Why can I `as[String]` with Ints? Is this type safe?

scala> (0 to 9).toDF("num").as[String]
res12: org.apache.spark.sql.Dataset[String] = [num: int]

Q2: Why can I map over strings even though there are really ints?

scala> (0 to 9).toDF("num").as[String].map(_.toUpperCase)
res11: org.apache.spark.sql.Dataset[String] = [value: string]

Why are the two lines possible?

Jacek Laskowski
Mastering Apache Spark 2.0
Follow me at

To unsubscribe e-mail:

View raw message