atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Madhan Neethiraj (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (ATLAS-442) Hive model to handle cascaded deletes
Date Mon, 18 May 2020 02:15:00 GMT

     [ https://issues.apache.org/jira/browse/ATLAS-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Madhan Neethiraj resolved ATLAS-442.
------------------------------------
    Resolution: Abandoned

> Hive model to handle cascaded deletes
> -------------------------------------
>
>                 Key: ATLAS-442
>                 URL: https://issues.apache.org/jira/browse/ATLAS-442
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Shwetha GS
>            Priority: Major
>
> Ideally, hive model should be like this:
> type - attributes
> hive_table - ..., partitions(list<hive_partition>), ...
> hive_partition - values(list<string>), ...
> partitions attribute in hive_table is composite. Add partition is frequent in hive, and
this would result in create new partition and add this reference to table. So, this results
in create entity + update array for hive_table entity. Since the number of partitions can
be large, we want to avoid updates to hive_table entity every time. 
> Hence hive model is implemented now as:
> hive_table - ...
> hive_partition - table(hive_table), values(list<string>), ...
> In this model, there is no way to indicate that partitions are composite attributes of
table and hence deleting table will nor result in deleting of partitions. But we need the
cascading delete of partitions in this case. We need to figure out a way to solve this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message