kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Murumkar <manoj.murum...@gmail.com>
Subject Re: Re: Using ACLs without Kerberos
Date Fri, 08 Sep 2017 19:44:26 GMT
This is fixed as well, btw. My ZK config wasn't done properly.

On Tue, Sep 5, 2017 at 11:08 PM, Manoj Murumkar <manoj.murumkar@gmail.com>
wrote:

> Thanks, SASL setup for ZK does the trick.
>
> Now, I am continuing the security setup with schema registry. I am
> configuring it to use same PLAINTEXT method for now.
>
> listeners=http://0.0.0.0:8081
> kafkastore.connection.url=localhost:2181
> kafkastore.topic=_schemas
> debug=true
> # Security specific params
> kafkastore.bootstrap.servers=SASL_PLAINTEXT://localhost:9092
> #kafkastore.bootstrap.servers=SASL_PLAINTEXT://or1010050208015:9092,SASL_
> PLAINTEXT://or1010050208016:9092,SASL_PLAINTEXT://
> or1010050208017:9092,SASL_PLAINTEXT://or1010050208018:9092
> kafkastore.init.timeout.ms=10000
> kafkastore.security.protocol=SASL_PLAINTEXT
> zookeeper.set.acl=true
> kafkastore.sasl.mechanism=PLAIN
> kafkastore.group.id=schema-registry-consumer-group
>
> I have followed instructions (modified for my use case and latest version
> of confluent) from here: https://www.confluent.io/blog/
> securing-confluent-schema-registry-apache-kafka/
>
> When I try to start schema registry, it fails with following error:
>
> [2017-09-06 06:05:23,790] ERROR Server died unexpectedly:
> (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:52)
> java.lang.IllegalArgumentException: Unable to subscribe to the Kafka
> topic _schemas backing this data store. Topic may not exist.
>     at io.confluent.kafka.schemaregistry.storage.
> KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:130)
>
>
> However, I have enabled schema registry user to produce/consume messages
> for _schemas topic and authorizer debug messages show the same:
>
> [2017-09-06 06:05:21,784] DEBUG operation = Read on resource =
> Topic:_schemas from host = 127.0.0.1 is Allow based on acl = User:s
> chemaregistry has Allow permission for operations: All from hosts: *
> (kafka.authorizer.logger)
> [2017-09-06 06:05:21,784] DEBUG Principal = User:schemaregistry is Allowed
> Operation = Describe from host = 127.0.0.1 on resource
> = Topic:_schemas (kafka.authorizer.logger)
> [2017-09-06 06:05:22,788] DEBUG operation = Read on resource =
> Topic:_schemas from host = 127.0.0.1 is Allow based on acl =
> User:schemaregistry has Allow permission for operations: All from hosts: *
> (kafka.authorizer.logger)
> [2017-09-06 06:05:22,788] DEBUG Principal = User:schemaregistry is Allowed
> Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas
> (kafka.authorizer.logger)
>
> Any ideas of what might be happening here?
>
> Here're contents of my JAAS file that I use in the startup script:
>
>
> [root@or1010050208015 kafka]# tail -1 /bin/schema-registry-start
> exec $(dirname $0)/schema-registry-run-class ${EXTRA_ARGS}
> -Djava.security.auth.login.config=/etc/schema-registry/
> schema_registry_client_jaas.properties io.confluent.kafka.
> schemaregistry.rest.SchemaRegistryMain "$@"
>
>
> [root@or1010050208015 kafka]# cat /etc/schema-registry/schema_
> registry_client_jaas.properties
> /* Kafka Client */
> KafkaClient {
>   org.apache.kafka.common.security.plain.PlainLoginModule required
>   username="schemaregistry"
>   password="schemaRegistry";
> };
>
> /* Zookeeper client */
> Client {
>     org.apache.zookeeper.server.auth.DigestLoginModule required
>     username="zk_admin"
>     password="zk_admin_secret";
> };
>
>
> On Thu, Aug 31, 2017 at 1:35 PM, Shrikant Patel <SPatel@pdxinc.com> wrote:
>
>> Yes, as per my understanding for now the only way to secure ZK is SASL.
>> https://issues.apache.org/jira/browse/ZOOKEEPER-2125 once released ZK
>> could also be secured using SSL.
>>
>> Also remember not use any OS user, anyone user on n\w who can connect to
>> ZK host:port will be able to modify the ACLs.
>>
>> - Shri
>>
>> -----Original Message-----
>> From: Manoj Murumkar [mailto:manoj.murumkar@gmail.com]
>> Sent: Thursday, August 31, 2017 11:03 AM
>> To: users@kafka.apache.org
>> Subject: [EXTERNAL] Re: Using ACLs without Kerberos
>>
>> ***** Notice: This email was received from an external source *****
>>
>> >> Current kafka-acls.sh script directly contacts zookeeper to create
>> ACLs.
>> Any OS user who got access to zookeeper can create ACLs for any Kafka
>> principal.
>>
>> Thanks for that point. Appreciate it.
>>
>> On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <manikumar.reddy@gmail.com>
>> wrote:
>>
>> > There is no correlation between OS user and Kafka Principal/Username.
>> > Here user name refers to the principal associated with the kafka
>> > communication channel (Kerberos Principal, SASL/Plain username, Scram
>> > username, SSL
>> > certificate)
>> >
>> > Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
>> > Any OS user who got access to zookeeper can create ACLs for any Kafka
>> > principal.
>> > This can be restricted by enabling Zk Kerberos authentication.
>> > http://kafka.apache.org/documentation/#zk_authz
>> >
>> > From Kafka 0.11.0.0, We can create ACLs using Java Admin API.
>> >
>> > On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar
>> > <manoj.murumkar@gmail.com>
>> > wrote:
>> >
>> > > Right, I am. Just to be clear, I am using kafka-acl script to
>> > define/remove
>> > > ACLs as a non-super user and it just works fine. I had expected it
>> > > to
>> > work
>> > > only for super users and not for regular users ('nex37045' is a
>> > > normal user).
>> > >
>> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
>> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
>> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
>> > > `Topic:test`:
>> > >      User:nex37045 has Deny permission for operations: Write from
>> hosts:
>> > *
>> > >     User:skumarmu has Allow permission for operations: Write from
>> hosts:
>> > *
>> > >
>> > > [nex37045@or1010051029033 ~]$
>> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
>> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
>> > > zookeeper.connect=localhost:2181 --remove --allow-principal
>> > User:skumarmu
>> > > --operation Write --topic test
>> > > Are you sure you want to remove ACLs:
>> > >      User:skumarmu has Allow permission for operations: Write from
>> > hosts: *
>> > >  from resource `Topic:test`? (y/n)
>> > > y
>> > > Current ACLs for resource `Topic:test`:
>> > >      User:nex37045 has Deny permission for operations: Write from
>> hosts:
>> > *
>> > >
>> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
>> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
>> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
>> > > `Topic:test`:
>> > >      User:nex37045 has Deny permission for operations: Write from
>> hosts:
>> > *
>> > >
>> > >
>> > >
>> > > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar
>> > > <manikumar.reddy@gmail.com>
>> > > wrote:
>> > >
>> > > > Looks like you are already using SASL/PLAIN mechanism. Kafka
>> > > > supports
>> > > SASL
>> > > > authentication framework.
>> > > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms.
>> > > > you
>> > can
>> > > > enable SSL encryption also
>> > > >
>> > > > http://kafka.apache.org/documentation.html#security
>> > > >
>> > > >
>> > > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <
>> > > manoj.murumkar@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Thanks Manikumar. I am testing the setup documented here:
>> > > > > https://developer.ibm.com/opentech/2017/05/31/kafka-
>> > acls-in-practice/
>> > > > > (SASL_PLAINTEXT).
>> > > > >
>> > > > > I haven't setup any authentication for the tests. Thinking about
>> > > > > it, authentication is a must have for authorization (so, kafka
>> > > > > knows
>> > who's
>> > > > > making resource request), right?
>> > > > >
>> > > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <
>> > manikumar.reddy@gmail.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi,
>> > > > > >
>> > > > > > Kafka default authorizer is used with secure authenticated
>> > > > > > channels (SSL,SASL,SCRAM).
>> > > > > > For plain text (non-secure) channels, the principal will
be
>> > > > > > always ANONYMOUS. Here you can authorize by ip-address.
>> > > > > >
>> > > > > > It's adviced to run on secure channels. you can try SASL/PLAIN
>> > > > > > or
>> > > SCRAM
>> > > > > > mechanisms with/without SSL.
>> > > > > > Pl, check Kafka docs for security considerations.
>> > > > > >
>> > > > > > >> I was testing if a non-admin principal (OS user)
can modify
>> > > > > (add/remove)
>> > > > > > ACLs and
>> > > > > > it seems like it's possible.
>> > > > > > How are you trying? Can you give more details about your
setup?
>> > What
>> > > > > > authentication mechanism, etc.?
>> > > > > >
>> > > > > >
>> > > > > > Thanks,
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
>> > > > > manoj.murumkar@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi,
>> > > > > > >
>> > > > > > > We are evaluating how to put authorization in place
for
>> > > > > > > Kafka
>> > > (around
>> > > > > > > topics, mostly). Is it a good idea to do this without
>> > > > > > > Kerberos? I
>> > > was
>> > > > > > > testing if a non-admin principal (OS user) can modify
>> > (add/remove)
>> > > > ACLs
>> > > > > > and
>> > > > > > > it seems like it's possible. If this is right behavior,
it's
>> > > insecure
>> > > > > and
>> > > > > > > unusable. What do you guys think?
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > >
>> > > > > > > Manoj
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> This e-mail and its contents (to include attachments) are the property of
>> National Health Systems, Inc., its subsidiaries and affiliates, including
>> but not limited to Rx.com Community Healthcare Network, Inc. and its
>> subsidiaries, and may contain confidential and proprietary or privileged
>> information. If you are not the intended recipient of this e-mail, you are
>> hereby notified that any unauthorized disclosure, copying, or distribution
>> of this e-mail or of its attachments, or the taking of any unauthorized
>> action based on information contained herein is strictly prohibited.
>> Unauthorized use of information contained herein may subject you to civil
>> and criminal prosecution and penalties. If you are not the intended
>> recipient, please immediately notify the sender by telephone at
>> 800-433-5719 or return e-mail and permanently delete the original e-mail.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message