ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Don Bosco Durai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (RANGER-768) Hive Metastore Plugin
Date Tue, 12 Jan 2016 04:47:39 GMT

    [ https://issues.apache.org/jira/browse/RANGER-768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15093288#comment-15093288
] 

Don Bosco Durai commented on RANGER-768:
----------------------------------------

These are good points.
bq.  For the latter, an exhaustive search will probably be launched to find all to-be-affected
derived HDFS policies and make adjustments on them if appropriate.
I feel, we should just sync up the folders/files associated with the Hive tables in Ranger
with the mapping. So in this way, we can virtually create the HDFS policies based on the Hive
policies. Madhan suggestion is to create the mapping separately, because there could be thousands
of partitions/folders/files per Hive table. So we might have to handle slight differently,
so we can scale without overloading the system.

bq. Note that this exhaustive search will be performed at policy modification and not at query
time for the sake of performance.
We have to remember that the partitions would be created after the policies are created. So
we need a hook in the Hive Metastores to send the events to Ranger. I thought that was your
original plan with the Listener. 

For your last 4 comments, the suggestion was the second plugin just send the change events.
Once the Ranger Admin gets the HDFS resources for the Hive table, it will recompute the implied
HDFS policies and send the updated policies to HDFS Ranger Plugin.

I think, a stage diagram might be good way to explain the flow of data. Let me see if I create
one. It will be easy to visualize.


Thanks


> Hive Metastore Plugin
> ---------------------
>
>                 Key: RANGER-768
>                 URL: https://issues.apache.org/jira/browse/RANGER-768
>             Project: Ranger
>          Issue Type: New Feature
>          Components: admin, plugins
>            Reporter: Yan
>         Attachments: Design Proposal for Hive Metastore Plugin of Ranger.docx, Design
Proposal for Hive Metastore Plugin of Ranger.docx
>
>
> Currently there is no Ranger processing of Hive table meta store events that could result
in privilege modifications. One example is that when a table is renamed by a Hive Server 2
client (the "beeline"), no proper privilege adjustments in Ranger are made to allow/deny previously
allowed/denied users the same privileges as before. In addition, more advanced features, such
as granting/denying similar accesses to Hive's HDFS data to users that have (or do not have)
privileges in the Hive, would require that detailed metadata of the Hive table, the storage
info to be specific, be available to Ranger in order to make the corresponding HDFS  data
accessible to the Hive users directly.
> This plugin will depend upon the existing Ranger Hive plugin, so it shares the same "service"
name as the associated Ranger Hive service deployed, and it will be "co-enabled" with the
existing Ranger Hive plugin.
> Design doc will come soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message