Hi,

Why is casting a string column to timestamp not giving the same results as going through casting to long in-between? I'm tempted to consider it a bug.

scala> spark.version
res4: String = 2.4.3

scala> Seq("1", "2").toDF("ts").select($"ts" cast "timestamp").show
+----+
|  ts|
+----+
|null|
|null|
+----+

scala> Seq("1", "2").toDF("ts").select($"ts" cast "long").select($"ts" cast "timestamp").show
+-------------------+
|                 ts|
+-------------------+
|1970-01-01 01:00:01|
|1970-01-01 01:00:02|
+-------------------+

Pozdrawiam,
Jacek Laskowski
----
The Internals of Spark SQL https://bit.ly/spark-sql-internals
The Internals of Spark Structured Streaming https://bit.ly/spark-structured-streaming
The Internals of Apache Kafka https://bit.ly/apache-kafka-internals