atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sarath Subramanian (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (ATLAS-3148) Implement Hive Metastore hook for Atlas
Date Wed, 17 Apr 2019 23:19:00 GMT

     [ https://issues.apache.org/jira/browse/ATLAS-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sarath Subramanian updated ATLAS-3148:
--------------------------------------
    Description: 
Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which registers with HiveServer2
process to listen for create/update/delete operations and updates the metadata in Atlas.

If hive metastore is accessed using other clients like Impala shell, Hue or other JDBC/ODBC
client apps there is no way to capture the events in Atlas.

This Jira will create a new atlas hook for Hive Metastore - stores the metadata for Hive
tables and partitions in a relational database, and provides clients (including Hive) access
to this information using the metastore service API. 

This hook is registered as post-listener (*hive.metastore.event.listeners*) and DDL operations
are captured and send to atlas kafka topic for processing by Atlas server.

The following DDL operations are captured:
 * *CreateDatabaseEvent*
 * *DropDatabaseEvent*
 * *AlterDatabaseEvent*
 * *CreateTableEvent*
 * *DropTableEvent*
 * *AlterTableEvent*

 

  was:
Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which registers with HiveServer2
process to listen for create/update/delete operations and updates the metadata in Atlas.

If hive metastore is accessed using other clients like Impala shell, Hue or other JDBC/ODBC
client apps there is no way to capture the events in Atlas.

This Jira will create a new atlas hook for Hive Metastore - stores the metadata for Hive
tables and partitions in a relational database, and provides clients (including Hive) access
to this information using the metastore service API. 

This hook is registered as post-listener (hive.metastore.event.listeners) and DDL operations
are captured and send to atlas kafka topic for processing by Atlas server.

The following DDL operations are captured:
 * CreateDatabaseEvent
 * DropDatabaseEvent
 * AlterDatabaseEvent
 * CreateTableEvent
 * DropTableEvent
 * AlterTableEvent

 


> Implement Hive Metastore hook for Atlas
> ---------------------------------------
>
>                 Key: ATLAS-3148
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3148
>             Project: Atlas
>          Issue Type: New Feature
>          Components:  atlas-core
>    Affects Versions: 1.1.0
>            Reporter: Sarath Subramanian
>            Assignee: Sarath Subramanian
>            Priority: Major
>             Fix For: 2.0.0, trunk
>
>
> Atlas has hive hook ([http://atlas.apache.org/Hook-Hive.html]) which registers with
HiveServer2 process to listen for create/update/delete operations and updates the metadata
in Atlas.
> If hive metastore is accessed using other clients like Impala shell, Hue or other JDBC/ODBC
client apps there is no way to capture the events in Atlas.
> This Jira will create a new atlas hook for Hive Metastore - stores the metadata for
Hive tables and partitions in a relational database, and provides clients (including Hive)
access to this information using the metastore service API. 
> This hook is registered as post-listener (*hive.metastore.event.listeners*) and DDL operations
are captured and send to atlas kafka topic for processing by Atlas server.
> The following DDL operations are captured:
>  * *CreateDatabaseEvent*
>  * *DropDatabaseEvent*
>  * *AlterDatabaseEvent*
>  * *CreateTableEvent*
>  * *DropTableEvent*
>  * *AlterTableEvent*
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message