ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pradeep Agrawal <pradeep.agra...@freestoneinfotech.com>
Subject Review Request 55565: RANGER-1308 : Remove DEFAULT CHARSET 'latin1' from ranger db schema script to enable support of unicode or any other charset
Date Mon, 16 Jan 2017 11:40:00 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, Madhan
Neethiraj, Mehul Parikh, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan

Bugs: RANGER-1308

Repository: ranger


**Problem Statement :** 
1. Currently ranger db schema script is creating most of the tables with latin1 charset. if
User/Group name with Unicode characters are not accepted in Ranger UI and from REST API actual
unicode character is replaced with "replacement character" (?).

2. The InnoDB storage engine has a maximum index length of 767 bytes, so for utf8 or utf8mb4
columns, we can index a maximum of 255 or 191 characters, respectively. If we currently have
utf8 columns with indexes longer than 191 characters, we will need to index a smaller number
of characters when using utf8mb4.

**Proposed Solution :** 
1. Remove hardcoded CHARSET 'latin1' from ranger db schema script ; so that user table shall
be created with utf8 charset if server charset is utf8.

2. Rather reducing width of index key to 191 or 255, we can enable 'innodb_large_prefix' in
my.cnf configuration(mysql server system variable configuration). When the 'innodb_large_prefix'
configuration option is enabled, the index key prefix length limit is raised to 3072 bytes
for InnoDB tables that use DYNAMIC or COMPRESSED row format. Since default ROW_FORMAT is COMPACT
we need to explicity add clause ROW_FORMAT=DYNAMIC in create table schema.


  kms/scripts/db/mysql/kms_core_db.sql 1177595 
  security-admin/db/mysql/patches/008-removeTrailingSlash.sql 2e00f5f 
  security-admin/db/mysql/patches/009-updated_schema.sql 78c7006 
  security-admin/db/mysql/patches/013-permissionmodel.sql ffd261b 
  security-admin/db/mysql/patches/016-updated-schema-for-tag-based-policy.sql b86ae10 
  security-admin/db/mysql/patches/020-datamask-policy.sql ac5f404 
  security-admin/db/mysql/patches/022-split-service-table.sql d99bfc1 
  security-admin/db/mysql/patches/025-create-schema-for-plugin-info.sql 85d1f0a 
  security-admin/db/mysql/xa_core_db.sql dcfce4d 

Diff: https://reviews.apache.org/r/55565/diff/


**Steps Performed(with patch):**
1. Installed MySQL 5.6 with the help of doc.
2. Untar ranger-admin from Build having changes of proposed patch.
3. Provided ranger db root and admin db details in install.properties.
4. Executed setup.sh script.
6. Tried to start ranger admin service.

**Expected behaviour :** Ranger admin should start normally and User should able to see Dashboard
page after login.

**Actual behaviour :** Ranger admin was started and was able to login and see Ranger UI.

7. Tried to create user with unicode characters in username field. for example (?,ΕΎ	,??,??
,????	)

**Expected behaviour :** Ranger user should get created with above mentioned unicode characters.

**Actual behaviour :** Ranger user were created successfully.


Pradeep Agrawal

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message