atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Mestry via Review Board <nore...@reviews.apache.org>
Subject Re: Review Request 71737: ATLAS-3477: Introduce purging of entity in Atlas
Date Thu, 14 Nov 2019 00:00:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71737/#review218626
-----------------------------------------------------------




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
Lines 159 (patched)
<https://reviews.apache.org/r/71737/#comment306423>

    Does this mean that if purge is requested, then even active edges are requested whereas
if not deleted edges are requested.
    
    Shouldn't it be that when purge is requested, both active and deleted edges are requested?



repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
Lines 200 (patched)
<https://reviews.apache.org/r/71737/#comment306424>

    Same comment as above.



webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
Lines 835 (patched)
<https://reviews.apache.org/r/71737/#comment306422>

    Duplicate code.


- Ashutosh Mestry


On Nov. 13, 2019, 9:04 p.m., Sidharth Mishra wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71737/
> -----------------------------------------------------------
> 
> (Updated Nov. 13, 2019, 9:04 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3477
>     https://issues.apache.org/jira/browse/ATLAS-3477
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> As part of this solution users can purge the entities which are already soft deleted.
This will be useful as someone may be interested to purge some list of soft deleted entities
from Atlas. Due to governance compliances, only admin can perform purge of entities.
> 
> Purge will also create separate Audit entry at HBase similar to delete entity operation.
> 
> 
> Diffs
> -----
> 
>   authorization/src/main/java/org/apache/atlas/authorize/AtlasPrivilege.java 59c596d66

>   client/client-v2/src/main/java/org/apache/atlas/AtlasClientV2.java 199d6bf58 
>   intg/src/main/java/org/apache/atlas/listener/EntityChangeListenerV2.java 444167eae

>   intg/src/main/java/org/apache/atlas/model/audit/EntityAuditEventV2.java e9cc7cd3d 
>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutationResponse.java 7ace00dff

>   intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java daf4ca88c 
>   repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListenerV2.java
43a9b842c 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
b94590bf1 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
d2544df4f 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityChangeNotifier.java
3389d2465 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasEntityStoreV2.java
17a96493f 
>   repository/src/main/java/org/apache/atlas/repository/store/graph/v2/AtlasGraphUtilsV2.java
4d57d8bc8 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 3d2a18f0c 
>   webapp/src/main/java/org/apache/atlas/notification/EntityNotificationListenerV2.java
48f0cd3c2 
>   webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java 464d46f1e 
>   webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java
5d506bbc2 
> 
> 
> Diff: https://reviews.apache.org/r/71737/diff/5/
> 
> 
> Testing
> -------
> 
> IT has been added. 
> Validated manually by running below mentioned queries and REST:
> 
> 
> a) create table thursday_07_11_1 (id int);
>    create table thursday_07_11_1_ctas as select * from thursday_07_11_1;
> 
>    create table thursday_07_11_2 (id int);
>    create table thursday_07_11_2_ctas as select * from thursday_07_11_2;
> b) The above will create two lineages. Go to Atlas UI and get the vertices for atleast
any two.
> 
> 
> c) Using rest delete those two entities or use below query:
> drop table thursday_07_11_1;
> drop table thursday_07_11_1_ctas;
> 
> drop table thursday_07_11_2;
> drop table thursday_07_11_2_ctas;
> 
> REST delete example:
> DELETE http://localhopst:31000/api/atlas/v2/entity/guid/7dd7fd9c-0185-41a5-aff0-4212fa7eb87c
> 
> d) Now use the same guids and call purge REST:
> DELETE http://localhost:31000/api/atlas/admin/purge/?guid=75983487-9e29-41ee-9bae-53f0a0b54c05&guid=98bee463-8810-402a-bcdc-3eb00bf50365
> 
> Check the response which will return all the successfully purged entities
> 
> e) For each guid check if the audit has separate new entry for purge. REST used:
> GET http://localhost:31000/api/atlas/v2/entity/7b6806c1-8daf-4e0f-8d10-672a192968f1/audit
> 
> Example Response:
> 
> [
>     {
>         "entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
>         "timestamp": 1573161789065,
>         "user": "admin",
>         "action": "ENTITY_PURGE",
>         "details": "Purged entity",
>         "eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161789065:1",
>         "entity": null,
>         "type": null
>     },
>     {
>         "entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
>         "timestamp": 1573161729972,
>         "user": "impala",
>         "action": "ENTITY_DELETE",
>         "details": "Deleted entity",
>         "eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161729972:1",
>         "entity": null,
>         "type": null
>     },
>     {
>         "entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
>         "timestamp": 1573161727305,
>         "user": "admin",
>         "action": "ENTITY_UPDATE",
>         "details": "Updated: {\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"replicatedTo\":null,\"replicatedFrom\":null,\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"description\":null,\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"status\":\"ACTIVE\",\"createdBy\":\"impala\",\"updatedBy\":\"admin\",\"createTime\":1573161723180,\"updateTime\":1573161727305,\"version\":0,\"labels\":[]}",
>         "eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161727305:0",
>         "entity": null,
>         "type": null
>     },
>     {
>         "entityId": "7b6806c1-8daf-4e0f-8d10-672a192968f1",
>         "timestamp": 1573161723180,
>         "user": "impala",
>         "action": "ENTITY_CREATE",
>         "details": "Created: {\"typeName\":\"hive_column\",\"attributes\":{\"owner\":\"admin\",\"qualifiedName\":\"default.thursday_07_11_5.id@cm\",\"name\":\"id\",\"comment\":null,\"position\":0,\"type\":\"int\"},\"guid\":\"7b6806c1-8daf-4e0f-8d10-672a192968f1\",\"isIncomplete\":false,\"provenanceType\":0,\"version\":0,\"relationshipAttributes\":{\"table\":{\"guid\":\"c5ae4503-b498-493c-86b5-d8d909c6b16a\",\"typeName\":\"hive_table\",\"uniqueAttributes\":{\"qualifiedName\":\"default.thursday_07_11_5@cm\"},\"relationshipType\":\"hive_table_columns\"}},\"proxy\":false}",
>         "eventKey": "7b6806c1-8daf-4e0f-8d10-672a192968f1:1573161723180:2",
>         "entity": null,
>         "type": null
>     }
> ]
> 
> 
> Thanks,
> 
> Sidharth Mishra
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message