kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sönke Liebau <soenke.lie...@opencore.com.INVALID>
Subject Re: Restricting client access to zookeeper metadata
Date Tue, 27 Feb 2018 04:05:06 GMT
Hi Reema, hi Naresh,

I'll try and answer both your questions together by expanding on the
topic a bit. Also, rereading my message I realize, that I phrased that
somewhat ambiguously, since a few of the terms in there are
overloaded.

First of, if you are using the java consumer or producer (which you
most probably are) then there is no need for these to have access to
the Zookeeper nodes. Only the old scala clients needed to talk to
Zookeeper. This allows you to firewall your Zookeeper cluster so that
only Kafka brokers can connect to them.

Moving on to the topic of listing topics things become a bit more
complex because both things are possible. If you run the shell command
"kafka-topics --list" that will connect to Zookeeper and retrieve a
list of topics. And this is black and white, you either see all topics
when you can access Zookeeper or none if you can't.
There is also the Java Admin Client that can list topics and this
talks to a Kafka broker to retrieve the topics. For this case, ACLs
apply and you will only see the topics you are allowed to access. The
main drawback of this method is, that there is no command line tool
for this yet, it is "just" a java api.

When I said "access the Kafka nodes" I meant being able to connect to
the Kafka brokers port on those machines, that would be enough to use
the java admin client as described above.

Hope this helps.

Best regards,
Sönke


On Mon, Feb 26, 2018 at 5:25 PM, naresh Goud <nareshgoud.dulam@gmail.com> wrote:
> It should require zookeeper connection always, because intern kafka brokers
> will interact with zookeeper for all meta data about topics.
> But its interesting, how would you give departments to access to kafka nodes
>
> @Sönke,
>
> Could you please shade some light on giving departements access to kafka
> nodes.? Is it like  departments able to ssh to kafka nodes and run describe
> command? so it will show topics metadata only topics in that node?
>
> Apologies, if my question is very basic.
>
> Thank you,
> Naresh
>
>
>
> Thanks,
> Naresh
> www.linkedin.com/in/naresh-dulam
> http://hadoopandspark.blogspot.com/
>
>
> On Mon, Feb 26, 2018 at 5:30 PM, Reema Chugani <reemachugani@outlook.com>
> wrote:
>
>> Hi Sönke,
>>
>> Thanks for the info, it is helpful!
>>
>> I can modify so that the departments can only access the Kafka nodes
>> themselves. However how would the consumers connect to the topics then?
>> Don't the consumer clients require to connect via Zookeeper?
>>
>> Thanks,
>> Reema
>>
>> On Fri, Feb 23, 2018 at 10:50 PM, Sönke Liebau <soenke.liebau@opencore.com
>> .invalid<mailto:soenke.liebau@opencore.com.invalid>> wrote:
>> Hi Reema,
>>
>> if your departments have access to Zookeeper then there probably is not
>> much you can do about them accessing data on other departments topics. I
>> assume that you have enabled Zookeeper ACLs, but even with those in place,
>> the topic metadata is world readable, so listing topics can be done by
>> anyone who has access to Zookeeper.
>>
>> If your departments can only access the Kafka nodes themselves then the
>> DESCRIBE action on Topics is I believe what you are looking for, without an
>> ACL in place to grant this, the topic should not be listed in Metadata
>> responses.
>>
>> I hope that helps, if you need more information let us know!
>>
>> Best regards,
>> Sönke
>>
>> Am 24.02.2018 06:32 schrieb "Reema Chugani" <reemachugani@outlook.com<
>> mailto:reemachugani@outlook.com>>:
>>
>> Hi,
>>
>> I am using Kafka 0.10.2.
>>
>> I have multiple topics & consumers set up with ACLS such that consumer can
>> only read from a particular topic. I am wondering how I can prevent a
>> consumer from accessing metadata in zookeeper about other topics? i.e,
>> prevent consumers from listing or getting info about topics in the cluster.
>> (Not let marketing dept see the topic names of finance topics.)
>>
>> Thanks,
>> Reema
>>
>>
>>



-- 
Sönke Liebau
Partner
Tel. +49 179 7940878
OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany

Mime
View raw message