ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhay Kulkarni (JIRA)" <j...@apache.org>
Subject [jira] [Created] (RANGER-2341) Support for Incremental policy updates to improve performance of ranger-admin and plugins by optimal building of policy-engine
Date Fri, 01 Mar 2019 01:07:00 GMT
Abhay Kulkarni created RANGER-2341:
--------------------------------------

             Summary: Support for Incremental policy updates to improve performance of ranger-admin
and plugins by optimal building of policy-engine
                 Key: RANGER-2341
                 URL: https://issues.apache.org/jira/browse/RANGER-2341
             Project: Ranger
          Issue Type: Improvement
          Components: Ranger
    Affects Versions: master
            Reporter: Abhay Kulkarni
            Assignee: Abhay Kulkarni
             Fix For: master


Requirements:
Currently, every change to any policy causes rebuilding of policy-engine from scratch. There
are several disadvantages:
1. Compute time for rebuilding
2. Large traffic from ranger-admin to each of the plugins
3. Large demand on JVM memory system resulting in frequent garbage collection and pauses of
JVM.
It will be more optimal to communicate only the changes and apply them to existing policy-engine.

Design notes:

Policy changes are logged into a new database table.
Cache management in ranger-admin is enhanced to use this table to figure out changes using
a previously known version number (provided by module requesting updated policies).
Policy engine supports update operation that accepts policy-deltas and returns a new policy
engine with deltas applied.
Resource Trie structures are copied from older policy-engine selectively, and not rebuilt
from scratch.
Backward compatibility is maintained with older plugins by adding another parameter to REST
API for downloading policies.
Ranger admin as well as component plugins may be configured to optionally use policy deltas
for its internal policy-engines. Policy deltas are disabled by default. In ranger-admin, policy-deltas
are enabled in the ranger-admin by setting configuration variable 'ranger.admin.supports.policy.deltas'
to true. In individual plugins, policy-deltas are enabled by setting configuration variable
'ranger.plugin.<service-type>.policy.rest.supports.policy.deltas' to "true".
Policy delta table is cleared of records older than a week on restart of ranger-admin.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message