ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhay Kulkarni <akulka...@hortonworks.com>
Subject Review Request 68741: De-normalize schema for storing tags and related objects
Date Mon, 17 Sep 2018 23:22:56 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68741/
-----------------------------------------------------------

Review request for ranger, Madhan Neethiraj, Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-2219
    https://issues.apache.org/jira/browse/RANGER-2219


Repository: ranger


Description
-------

Currently, tag-definitions, tags and service-resources are stored in database using a normalized
form. When constructing resource->tag mappings, this schema design may lead to a large
number of database accesses, thereby causing a major performance bottleneck when the number
of resource->tag associations is large.

Denormalized schema will reduce the number of database accesses, and improve overall performance
significantly.


Diffs
-----

  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerServiceResource.java 67230c6de

  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerTag.java 9e620c32c 
  agents-common/src/main/java/org/apache/ranger/plugin/model/RangerTagDef.java c787beca5 
  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 3f23b0082 
  security-admin/db/mysql/patches/040-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql bafdb9676 
  security-admin/db/oracle/patches/040-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 2bc58acfe 
  security-admin/db/postgres/patches/040-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 1b64eeac8

  security-admin/db/sqlanywhere/patches/040-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 4a216fed9 
  security-admin/db/sqlserver/patches/040-denormalize-tag-tables.sql PRE-CREATION 
  security-admin/scripts/db_setup.py 02701c726 
  security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java 7875bc2a8 
  security-admin/src/main/java/org/apache/ranger/biz/TagDBStore.java d29df930c 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceDao.java ee0e40043 
  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementDao.java c9a1c2132

  security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementValueDao.java
364af6d0f 
  security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDao.java 40c3a887b 
  security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDefDao.java 129f3c13c 
  security-admin/src/main/java/org/apache/ranger/db/XXTagDao.java e14f836e7 
  security-admin/src/main/java/org/apache/ranger/db/XXTagDefDao.java c0dd88371 
  security-admin/src/main/java/org/apache/ranger/entity/XXServiceResource.java 961627a3c 
  security-admin/src/main/java/org/apache/ranger/entity/XXTag.java d26a0b079 
  security-admin/src/main/java/org/apache/ranger/entity/XXTagDef.java 818908ba8 
  security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java
PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/rest/ServiceTagsProcessor.java 9c19bb032

  security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceService.java
d7256802a 
  security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceServiceBase.java
6af682a81 
  security-admin/src/main/java/org/apache/ranger/service/RangerTagDefService.java 82eb252e6

  security-admin/src/main/java/org/apache/ranger/service/RangerTagService.java 28b9115fa 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml cdf6ba655 
  security-admin/src/test/java/org/apache/ranger/rest/TestServiceTagProcessor.java 952463318

  security-admin/src/test/java/org/apache/ranger/service/TestRangerTagDefService.java 8f19ffd78

  security-admin/src/test/java/org/apache/ranger/service/TestRangerTagService.java 1175989e2



Diff: https://reviews.apache.org/r/68741/diff/1/


Testing
-------

Tested using a local VM.

Verified that the x_tag_def, x_tag, x_service_resource tables are updated correctly (data
porting) when ranger-admin starts.
Verified that plugins receive correct tags by checking /etc/ranger/<service-name>/policycache/<service-type>_<service-name>_tag.json
file in policycache directory.
Verified that tags sent by ranger-tagsync are processed correctly by ranger-admin.


Thanks,

Abhay Kulkarni


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message