kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Don Bosco Durai <bo...@apache.org>
Subject Re: Authorization Engine For Kafka Related to KPI-11
Date Wed, 04 Nov 2015 18:33:55 GMT
Bhavesh

I am from the Apache Ranger team, so let me answer the questions pertaining to Ranger…

> 1) Is there any performance impact with Brokers/Producer/Consumer while using Apache
Ranger ?
As such Ranger overhead is negligible. Specifically for Kafka, we did some more optimization
for the unique usage pattern of Kafka consumers and publishers. Considering Kafka itself is
optimized for super throughput, expected some overhead penalty. We don’t have the exact
numbers yet. Would you be willing to help us some run some real-world scenarios to determine
the overhead? If so, we can discuss it in the Ranger mailing list. Thanks.

> 2) Is Audit log really useful out-of-box ? or let me know what sort of reports you run
on audit logs (e.g pumping Apache Ranger audit log into any other system for reporting purpose).
If you are referring to Ranger audit logs, then it is configurable. OOTB, it has options to
go to HDFS, SOLR and/or DB. We have extensions to send it to Kafka and any Log4J appender.
Audit logs has its own purpose, from compliance requirements to operation monitoring usage
to anomaly detection. Ranger portal has its own UI for querying. Since the audits are stored
in normalized format, you can write your own reports on top of it.

Bosco





On 11/3/15, 9:55 PM, "Bhavesh Mistry" <mistry.p.bhavesh@gmail.com> wrote:

>+ Kafka Dev team to see if Kafka Dev team know or recommend any Auth
>engine for Producers/Consumers.
>
>Thanks,
>
>Bhavesh
>
>Please pardon me,  I accidentally send previous blank email.
>
>On Tue, Nov 3, 2015 at 9:52 PM, Bhavesh Mistry
><mistry.p.bhavesh@gmail.com> wrote:
>> On Sun, Nov 1, 2015 at 11:15 PM, Bhavesh Mistry
>> <mistry.p.bhavesh@gmail.com> wrote:
>>> HI All,
>>>
>>> Have any one used Apache Ranger as Authorization Engine for Kafka Topic
>>> creation, consumption (read) and  write operation on a topic.  I am looking
>>> at having audit log and regulating consumption/ write to particular topic
>>> (for example, having production environment access does not mean that anyone
>>> can run console consumer etc on particular topic. Basically, regulate who
>>> can read/write to a topic as first use case).
>>>
>>> https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+0.5+-+User+Guide#ApacheRanger0.5-UserGuide-KAFKA
>>>
>>> If you have used Apache Ranger in production, I have following question:
>>> 1) Is there any performance impact with Brokers/Producer/Consumer while
>>> using Apache Ranger ?
>>> 2) Is Audit log really useful out-of-box ? or let me know what sort of
>>> reports you run on audit logs (e.g pumping Apache Ranger audit log into any
>>> other system for reporting purpose).
>>>
>>> Please share your experience using Kafka with any other Authorization engine
>>> if you are not using Apache Ranger (This is based on
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-11+-+Authorization+Interface).
>>>
>>> Thanks and looking forward to hear back from Apache Kafka Community members.
>>>
>>> Thanks,
>>>
>>> Bhavesh


Mime
View raw message