ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Don Bosco Durai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (RANGER-1780) Allow AuditSummaryQueue to aggregate events in the same directory
Date Tue, 26 Sep 2017 00:03:00 GMT

    [ https://issues.apache.org/jira/browse/RANGER-1780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179979#comment-16179979
] 

Don Bosco Durai commented on RANGER-1780:
-----------------------------------------

Yes, I agree, it will be good if we can come up with some heuristics, regex or plain flat
file to compute the key.

Configuration for plugins is independent for each component. So you can selectively enable
this feature only for HDFS. Regardless, we should make this generic.

Resources in Kafka are topics and it is up to the user how they name it and also they don't
implicitly support hierarchy, even though there were discussions to support first-class multi-tenancy.
Today you can implement it by using dot notation and use Ranger's delegated admin feature.

YARN uses dot notation. While for Hive and HBase, we concatenate the resources to /database/table/column
format.

So the delimiter is application/component specific. So we have to keep this generic. Seems,
you are most interested in Hive database files, so you could consider a list of regex which
can extract the basepaths. E.g. if we give /user/hive/datawarehouse/mydb/mytable/myfile or
/user/hive/datawarehouse/mydb/mytable/mydir/myfile, then your function based on regex or some
lookup, should return "/user/hive/datawarehouse/mydb/mytable"




> Allow AuditSummaryQueue to aggregate events in the same directory
> -----------------------------------------------------------------
>
>                 Key: RANGER-1780
>                 URL: https://issues.apache.org/jira/browse/RANGER-1780
>             Project: Ranger
>          Issue Type: Improvement
>          Components: audit
>    Affects Versions: 0.7.1
>            Reporter: Alejandro Fernandez
>            Assignee: Alejandro Fernandez
>             Fix For: 1.0.0
>
>         Attachments: RANGER-1780.patch, ranger_summary.png
>
>
> AuditSummaryQueue already has logic to enable the summarization, but it requires 2 events
to have the exact same resource path (plus a couple of other fields such as user, access type,
access result, action, client ip, session).
> This Jira is to add a config called xasecure.audit.provider.summary.aggregate.level so
that if it is set to "directory" then 2 events can still be aggregated if they are files in
the same directory.
> If the config is not specified its default value will be "file" which preserves the existing
behavior.
> See [^ranger_summary.png] for screenshot on desired behavior.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message