ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Madhan Neethiraj" <mad...@apache.org>
Subject Review Request 40160: RANGER-702: optimize policy download performance
Date Tue, 10 Nov 2015 23:47:54 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40160/
-----------------------------------------------------------

Review request for ranger, Alok Lal, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Ramesh
Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.


Bugs: RANGER-702
    https://issues.apache.org/jira/browse/RANGER-702


Repository: ranger


Description
-------

Optimize the code that retrieves policies from the database - to minimize the number of queries
to the database. This is an update over review 39475 - to handle failure scenarios gracefully.
Added PERF logs to public methods in RangerPolicyRetriever and ServiceREST classes; to enable
PERF logs, configure log4j to set INFO level for "ranger.perf.".


Diffs
-----

  agents-common/src/main/java/org/apache/ranger/plugin/util/RangerPerfTracer.java PRE-CREATION

  security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java PRE-CREATION

  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 7dbf9fb 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemAccessDao.java d9952e0 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemConditionDao.java 4ed59f1

  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemDao.java 9c3fe2e 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemGroupPermDao.java fe0c806

  security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemUserPermDao.java d8235bc 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceDao.java 0cdb75e 
  security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceMapDao.java ecf4da0 
  security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java ff4d297 
  security-admin/src/main/java/org/apache/ranger/service/RangerBaseModelService.java 9ecd77e

  security-admin/src/main/java/org/apache/ranger/service/RangerPolicyService.java 12367e6

  security-admin/src/main/java/org/apache/ranger/service/RangerPolicyServiceBase.java 8c73c4c

  security-admin/src/main/java/org/apache/ranger/service/RangerPolicyWithAssignedIdService.java
0dae4bd 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml 4de99e2 
  security-admin/src/test/java/org/apache/ranger/service/TestRangerPolicyService.java dc375f5


Diff: https://reviews.apache.org/r/40160/diff/


Testing
-------

After this updates, it took less than 10 seconds to download ~1500 Hive policies; earlier
this took more than 10 minutes to download.


Thanks,

Madhan Neethiraj


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message