nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <johannes.meix...@post.ch>
Subject Improving Ranger audit logging in Nifi
Date Mon, 07 May 2018 12:50:13 GMT
We're using Ranger to provide LDAP group-based authorization to NiFi resources via the Ranger-NiFi
plugin. This works well, and read-only / read-write access can be defined quite granularly.

Audit logs are written into Solr in order for us to search them using Ranger GUI later on.
This worked fine until Friday, upon which "something changed" (NiFi was restarded without
changes) and the connection between nifi-ranger-plugin and Solr was interrupted.

Today, then, Nifi's Heap Usage went through the roof up to 97%, most of which ConcurrentHashMaps
spawned by ManagedRangerAuthorizer.

We've noticed this type of memory leak in the past: previously, ranger-nifi-plugin failed
to send resources to Solr due to a mismatch in SSL FQDNs.

This could be solved by setting SOLR_HOST=`hostname -f` in the infra-solr-env template, but
the logging of the ManagedRangerAuthorizer and its batch transmitting to Solr is rather...
shall we say... parsimonious.

How do I increase it so I can actually see what's causing the memory leak in Ranger/NiFi/Solr
interactions?


Regards,



Johannes Meixner

Post CH AG
Informationstechnologie
DWH & Big Data Solutions
Webergutstrasse 5
3030 Bern (Zollikofen)

Internet: http://www.post.ch
Tel.: +41 58 341 40 77


Mime
View raw message