spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (Jira)" <>
Subject [jira] [Updated] (SPARK-27882) Support SQL:2016 compatible datetime patterns
Date Mon, 16 Mar 2020 22:53:04 GMT


Dongjoon Hyun updated SPARK-27882:
    Affects Version/s:     (was: 3.0.0)

> Support SQL:2016 compatible datetime patterns
> ---------------------------------------------
>                 Key: SPARK-27882
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Peter Toth
>            Priority: Major
> Date and time related functions in Spark use [DateTimeFormatter|]
to parse and format dates where a pattern is defined. Unfortunately, it is not entirely compatible
with the SQL:2016 standard.
> There is an initiative in Impala: IMPALA-4018 and Hive HIVE-21575 to support SQL:2016
patterns as both these systems lack this feature currently.
>  The [Impala design document|]
and [compatibility matrix|]
created by [~gaborkaszab] covers how the compliance with the standard can be achieved and
how do other databases support this standard. I believe these documents are very useful for
Spark community as well.
>  Since Hive is also JVM based I also see an option to share some code that does the parsing/formatting
according to the standard.
> (Please note that format patterns are just one part of these tickets, the other part
about `CAST (... FORMAT <pattern>)` function, which is also part of the SQL:2016 standard,
is covered here: SPARK-27881)

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message