ranger-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alok Lal" <a...@hortonworks.com>
Subject Re: Review Request 39101: Ranger to support Azure Blob Datastore as an audit destination via HDFS audit handler
Date Thu, 08 Oct 2015 23:27:11 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39101/
-----------------------------------------------------------

(Updated Oct. 8, 2015, 4:27 p.m.)


Review request for ranger, Don Bosco Durai, Gautam Borad, Madhan Neethiraj, and Velmurugan
Periasamy.


Changes
-------

Addressed rework comments:
- Move config-map to base class
- Added support to both V2 and V3 audit libraries
- Empty properties would not be added to config so that install is simplified

Also made change to allow for manual install.  Changes in install.properties and audit-change.cfg
files.
- Change to install framework is needed because azure support needs properties whose name
changes based on account name used.


Bugs: RANGER-682
    https://issues.apache.org/jira/browse/RANGER-682


Repository: ranger


Description
-------

Hadoop has added native support for Azure Blob Storage.  Ranger allows to write audit messages
to it via Hadoops support for writing to Azure Blob Storage by existing HDFS Audit Provider.
- Also fixed a issue to reconnect to Filesystem in case of error
- Configuration support added only for v3 style of configuration.
- This patch applies to master.  However, it will be ported to Ranger-0.5 branch, too.


Diffs (updated)
-----

  agents-audit/src/main/java/org/apache/ranger/audit/destination/HDFSAuditDestination.java
49e5fbb 
  agents-audit/src/main/java/org/apache/ranger/audit/provider/BaseAuditHandler.java 30db18b

  agents-audit/src/main/java/org/apache/ranger/audit/provider/hdfs/HdfsAuditProvider.java
a18e3e9 
  agents-audit/src/main/java/org/apache/ranger/audit/provider/hdfs/HdfsLogDestination.java
49f4e65 
  agents-installer/src/main/java/org/apache/ranger/utils/install/XmlConfigChanger.java 05fbb23

  hbase-agent/conf/ranger-hbase-audit-changes.cfg b540e18 
  hbase-agent/scripts/install.properties 9170f41 
  hdfs-agent/conf/ranger-hdfs-audit-changes.cfg 93e7b86 
  hdfs-agent/scripts/install.properties 289117b 
  hive-agent/conf/ranger-hive-audit-changes.cfg daa8b74 
  hive-agent/scripts/install.properties 9370600 
  kms/scripts/install.properties 563a0e0 
  knox-agent/conf/ranger-knox-audit-changes.cfg 57858f7 
  knox-agent/scripts/install.properties 2fee7ee 
  plugin-kafka/conf/ranger-kafka-audit-changes.cfg 0445576 
  plugin-kafka/scripts/install.properties 1e2854e 
  plugin-kms/conf/ranger-kms-audit-changes.cfg f7d3543 
  plugin-solr/conf/ranger-solr-audit-changes.cfg 7e77a30 
  plugin-solr/scripts/install.properties 1f3852e 
  plugin-yarn/conf/ranger-yarn-audit-changes.cfg bfc2cd8 
  plugin-yarn/scripts/install.properties 9358033 
  storm-agent/conf/ranger-storm-audit-changes.cfg bfc2cd8 
  storm-agent/scripts/install.properties 0cee655 

Diff: https://reviews.apache.org/r/39101/diff/


Testing (updated)
-------

Write to Azure blob storage via TestEvents utlity
- Microsoft HDInisight currently clusters don't have support for kerberos.  However, testing
was done with and without wire encryption (WASB and WASBS schemas).
- Tested with both V3 and V2 libraries
- Did manual install of hbase plugin to confirm that ranger-hbase-audit.xml file gets created
with right property names and values.


Thanks,

Alok Lal


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message