atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-3539) Difference between hive_db information sent to ATLAS_HOOK created by beeline vs spark-shell
Date Thu, 28 Nov 2019 18:32:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-3539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984594#comment-16984594
] 

ASF subversion and git services commented on ATLAS-3539:
--------------------------------------------------------

Commit 07bd944c7d9eeaa23323c61e1d567e51132b1e55 in atlas's branch refs/heads/branch-2.0 from
Mandar Ambawane
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=07bd944 ]

ATLAS-3539: fixed Hive metastore hook to populate hive_db.ownerType, and not update hive_db.description
when null

Signed-off-by: Madhan Neethiraj <madhan@apache.org>
(cherry picked from commit c6661ff15c15856ae030c95ffbbf808ff0850d5a)


> Difference between hive_db information sent to ATLAS_HOOK created by beeline vs spark-shell
> -------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-3539
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3539
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Mandar Ambawane
>            Assignee: Mandar Ambawane
>            Priority: Major
>
> When HiveDB created by beeline client, ATLAS_HOOK recieves following json
> {code:java}
> {"version":{"version":"1.0.0","versionParts":[1]},"msgCompressionKind":"NONE","msgSplitIdx":1,"msgSplitCount":1,"msgSourceIP":"10.65.34.200","msgCreatedBy":"hive","msgCreationTime":1574834575732,"message":{"type":"ENTITY_CREATE_V2","user":"hive","entities":{"referredEntities":{},"entities":[{"typeName":"hive_db","attributes":{"owner":"hive","ownerType":"USER","qualifiedName":"demo_hive@cm","clusterName":"cm","name":"demo_hive","description":null,"location":"hdfs://mandarambawane-maatlas1120-1.vpc.cloudera.com:8020/warehouse/tablespace/managed/hive/demo_hive.db","parameters":{}},"guid":"-583477356828349","isIncomplete":false,"provenanceType":0,"version":0,"proxy":false}]}}} {code}
>  
> When HiveDB created by spark client, ATLAS_HOOK recieves following json 
> {code:java}
> {{ "version": { "version": "1.0.0", "versionParts": [1] }, "msgCompressionKind": "NONE",
"msgSplitIdx": 1, "msgSplitCount": 1, "msgSourceIP": "10.65.34.200", "msgCreatedBy": "hive",
"msgCreationTime": 1574834794032, "message": { "type": "ENTITY_CREATE_V2", "user": "spark",
"entities": { "referredEntities": {}, "entities": [{ "typeName": "hive_db", "attributes":
{ "owner": "spark", "qualifiedName": "demo_spark@cm", "clusterName": "cm", "name": "demo_spark",
"description": "", "location": "hdfs://mandarambawane-maatlas1120-1.vpc.cloudera.com:8020/warehouse/tablespace/managed/hive/demo_spark.db",
"parameters": {} }, "guid": "-583477356828350", "isIncomplete": false, "provenanceType": 0,
"version": 0, "proxy": false }] } }}{code}
>  
> Attribute "ownerType" is missing from the json, when we create db from spark.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message