ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhay Kulkarni <akulka...@hortonworks.com>
Subject Re: Review Request 70682: RANGER-2420: Ranger spends 36% of CPU in ObjectMapper
Date Tue, 28 May 2019 19:07:27 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70682/
-----------------------------------------------------------

(Updated May 28, 2019, 7:07 p.m.)


Review request for ranger, Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, and Velmurugan
Periasamy.


Changes
-------

Instead of using process-wide unique ObjectMapper, create a new ObjectMapper for every thread.


Bugs: RANGER-2420
    https://issues.apache.org/jira/browse/RANGER-2420


Repository: ranger


Description
-------

Ranger uses ObjectMapper to convert to/from JSON. In a profile of a workload (an Impala authorization
test) I see that 36% of the ranger CPU is spent in these functions. 26% of total CPU is in
the '_findRootDeserializer' method, which gets cached if a type is deserialized multiple times.
However, the caching is only effective if the ObjectMapper is reused. JSONUtil appears to
create a new ObjectMapper on every call, which defeats the caching.

Patch ensures that there is only one instance of ObjectMapper created in a process.


Diffs (updated)
-----

  agents-common/src/main/java/org/apache/ranger/admin/client/datatype/GrantRevokeData.java
a8d3c6af5 
  agents-common/src/main/java/org/apache/ranger/admin/client/datatype/RESTResponse.java 333f56504

  agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java
5e2c49211 
  agents-common/src/main/java/org/apache/ranger/plugin/util/JsonUtilsV2.java PRE-CREATION

  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java bc692937b 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSJSONReader.java d3e0064db 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSJSONWriter.java 3674e7a87 
  knox-agent/src/main/java/org/apache/ranger/services/knox/client/KnoxClient.java 0c83ef9bb

  security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java 35dc9405b 
  security-admin/src/main/java/org/apache/ranger/common/JSONUtil.java 5bc1baf58 
  security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java b5caea8ef 
  security-admin/src/main/java/org/apache/ranger/service/RangerDataHistService.java a96d648a9

  security-admin/src/main/java/org/apache/ranger/service/RangerPolicyService.java 411744237

  security-admin/src/main/java/org/apache/ranger/service/XAssetService.java 5525584f0 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java ac9af5eb4 
  security-admin/src/test/java/org/apache/ranger/service/TestRangerDataHistService.java e1a236e5f



Diff: https://reviews.apache.org/r/70682/diff/4/

Changes: https://reviews.apache.org/r/70682/diff/3-4/


Testing
-------

Ran all unit tests successfully.


Thanks,

Abhay Kulkarni


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message