spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-30112) Insert overwrite should be able to overwrite to same table under dynamic partition overwrite
Date Fri, 06 Dec 2019 17:23:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-30112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dongjoon Hyun resolved SPARK-30112.
-----------------------------------
    Fix Version/s: 3.0.0
       Resolution: Fixed

Issue resolved by pull request 26752
[https://github.com/apache/spark/pull/26752]

> Insert overwrite should be able to overwrite to same table under dynamic partition overwrite
> --------------------------------------------------------------------------------------------
>
>                 Key: SPARK-30112
>                 URL: https://issues.apache.org/jira/browse/SPARK-30112
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: L. C. Hsieh
>            Assignee: L. C. Hsieh
>            Priority: Major
>             Fix For: 3.0.0
>
>
> Currently, Insert overwrite cannot overwrite to same table even it is dynamic partition
overwrite. But for dynamic partition overwrite, we do not delete partition directories ahead.
We write to staging directories and move data to final partition directories. We should be
able to insert overwrite to same table under dynamic partition overwrite.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message