ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 42601: RANGER-798 - Approach 1 : Handle different timezone issue while saving audit logs to Solr
Date Thu, 28 Jan 2016 08:32:11 GMT

This is an automatically generated e-mail. To reply, visit:

agents-audit/src/main/java/org/apache/ranger/audit/destination/DBAuditDestination.java (line

    Instead of cloning and updating the eventDate in every destination (db/hdfs/solr/log4j),
consider sending the "local" time (as set in AuthzAuditEvent.eventTime) to all audit destinations
- exception DB, which require the time in UTC. For DB, consider updating the time in AuthzAuditEventDbObj(AuthzAuditEvent
event) constructor.

- Madhan Neethiraj

On Jan. 27, 2016, 4:21 a.m., Gautam Borad wrote:
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42601/
> -----------------------------------------------------------
> (Updated Jan. 27, 2016, 4:21 a.m.)
> Review request for ranger, Alok Lal, Don Bosco Durai, Abhay Kulkarni, Madhan Neethiraj,
Ramesh Mani, Selvamohan Neethiraj, and Velmurugan Periasamy.
> Bugs: RANGER-798
>     https://issues.apache.org/jira/browse/RANGER-798
> Repository: ranger
> Description
> -------
> **Problem Statement**:
> The current implementation of “auditEvent.getEventTime()” contains time in UTC and
since it's a date object it will contain component machine's local timezone. When Solr receives
this date object and timezone, it tries to convert it from given timezone to UTC timestamp,
which leads to double conversion of actual time before it get stored in Solr.
> **Proposed Solution**:
> If we can provide server local time and timezone to Solr then Solr will convert the received
time from given timezone to UTC.
> As an alternate solution, replaced getUTCDate() with new Date() object at various places
for audit event time, all audit destination will  receive local Date object, for Solr there
will be no conversion on received Date object but for all other audit destination we need
to convert the received Date value to UTC timestamp as audit logs are being stored in UTC
timestamp for all service/component. If all destination thread are enabled then changing the
received event object may create issue in other audit destination as same event object is
refferred everywhere. Hence received event object attributes value are being copied in another
local event object and the updated event time can be stored there, after this local event
object will be used to convert that in JSON to write in HDFS, or can be persisted in DB.
> Diffs
> -----
>   agents-audit/src/main/java/org/apache/ranger/audit/destination/DBAuditDestination.java
>   agents-audit/src/main/java/org/apache/ranger/audit/destination/FileAuditDestination.java
>   agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
>   agents-audit/src/main/java/org/apache/ranger/audit/destination/Log4JAuditDestination.java
>   agents-audit/src/main/java/org/apache/ranger/audit/model/AuditEventBase.java 2a07e94

>   agents-audit/src/main/java/org/apache/ranger/audit/provider/BaseAuditHandler.java 6717c92

>   agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java 9586f73 
>   agents-common/src/main/java/org/apache/ranger/authorization/utils/StringUtil.java f6f3d2d

>   agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestImpl.java
>   hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java
>   hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>   hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAccessRequest.java
>   hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuditHandler.java
>   plugin-kafka/src/main/java/org/apache/ranger/authorization/kafka/authorizer/RangerKafkaAuthorizer.java
>   plugin-kms/src/main/java/org/apache/ranger/authorization/kms/authorizer/RangerKmsAuthorizer.java
>   plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>   plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
> Diff: https://reviews.apache.org/r/42601/diff/
> Testing
> -------
> Steps performed(after patch) :
> 1) Changed plugin system time zone to IST and restarted all components.
> 2) Initiated an HDFS audit event.
> 3) Checked event time of newly created audit log in Solr, Audit log event time was matching
with UTC.
> 4) Checked event time in Ranger UI, newly generated Audit event is matching with current
> 5) Checked event time of newly created audit log in xa_access_audit table, Audit log
event time was matching with UTC.
> 6) Checked event time of newly created audit log in HDFS logs, Audit log event time was
matching with UTC.
> Note: Will test other services audit logs after this approach is reviewed.
> Thanks,
> Gautam Borad

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message