ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Luo <andrewluotechnolog...@outlook.com>
Subject Re: Review Request 72166: RANGER-2713: Remove audit fields from XXPolicyRef objects
Date Fri, 06 Mar 2020 04:29:33 GMT


> On March 3, 2020, 12:28 p.m., Pradeep Agrawal wrote:
> > From which version upgrade case has been tested ? can you try from ranger-0.7 also
if it was not tested?
> 
> Andrew Luo wrote:
>     I tested from the previous build (without my change) to a version with my change.
 I'll also test from 0.7 and let you know the results.
> 
> Andrew Luo wrote:
>     Done, I tested from 0.7.0 to my local build (on MySQL) and did a similar test:
>     
>     1. Install 0.7.0
>     2. Created new user
>     3. Logged in as new user, created a policy
>     4. Upgraded to my local build
>     5. Logged back in as original user, delete user that created policy
>     6. Verified no errors, policy is still present/correct
>     
>     I found some other bugs during the upgrade with the patches related to Kafka (J10015,
J10025, J10033) but the error was showing it was failing to insert into x_access_type_def.
 I didn't change that table in my patch so I think its a different issue (I will file a bug
later if I repro this on a build without my changes).  Anyways, during the upgrade I just
skipped those patches by manually inserting those records into the table so the upgrade script
would think those patches already got applied, and everything worked as expected.
> 
> Andrew Luo wrote:
>     (I also tested policy creation after upgrade)
> 
> Pradeep Agrawal wrote:
>     Thanks Andrew. I will also see what are the issues while upgrading from ranger-0.7
to ranger-2.1

I found the issue: https://reviews.apache.org/r/72205


- Andrew


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72166/#review219731
-----------------------------------------------------------


On Feb. 26, 2020, 10:18 p.m., Andrew Luo wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72166/
> -----------------------------------------------------------
> 
> (Updated Feb. 26, 2020, 10:18 p.m.)
> 
> 
> Review request for ranger and Pradeep Agrawal.
> 
> 
> Bugs: RANGER-2713
>     https://issues.apache.org/jira/browse/RANGER-2713
> 
> 
> Repository: ranger
> 
> 
> Description
> -------
> 
> XXPolicyRef objects have fields such as create time, update time, added by user ID, updated
by user ID, but there fields are entirely useless since they are all copied from the XXPolicy
object. In addition, while improving performance for creation of policies with large numbers
of users, we discovered that a lot of time was being spent in JPA converting these Date objects
especially. After removing these fields we saw a significant performance improvement (a secondary
benefit is less database space usage).
> 
> 
> Diffs
> -----
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 47618f6b3 
>   security-admin/db/mysql/patches/046-drop-audit-columns-from-policy-ref-tables.sql PRE-CREATION

>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql e59e7de61 
>   security-admin/db/oracle/patches/046-drop-audit-columns-from-policy-ref-tables.sql
PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 37ea61912

>   security-admin/db/postgres/patches/046-drop-audit-columns-from-policy-ref-tables.sql
PRE-CREATION 
>   security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql bd1c47cc4

>   security-admin/db/sqlanywhere/patches/046-drop-audit-columns-from-policy-ref-tables.sql
PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 22e1746f2

>   security-admin/db/sqlserver/patches/046-drop-audit-columns-from-policy-ref-tables.sql
PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java baacfa4d8

>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java 6af8f99f4

>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 4f4409d6a

>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
cb926740e 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 32a1b9f24

>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 115064621

>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 7aee502e0

>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 8dfb92833

>   security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java 85e457efa

>   security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 69c8a4cb6

> 
> 
> Diff: https://reviews.apache.org/r/72166/diff/1/
> 
> 
> Testing
> -------
> 
> Fresh install and upgrade tested on all 5 databases (tested service and policy creation)
> 
> Also tested user deletion on MySQL:
> 
> 1. Created new user
> 2. Logged in as new user, created a policy
> 3. Logged back in as original user, delete user that created policy
> 4. Verified no errors, policy is still present/correct
> 
> Also searched the source code and JPA queries xml for other references to these fields
> 
> 
> Thanks,
> 
> Andrew Luo
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message