kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Murumkar <manoj.murum...@gmail.com>
Subject Re: Re: Using ACLs without Kerberos
Date Wed, 06 Sep 2017 06:08:10 GMT
Thanks, SASL setup for ZK does the trick.

Now, I am continuing the security setup with schema registry. I am
configuring it to use same PLAINTEXT method for now.

listeners=http://0.0.0.0:8081
kafkastore.connection.url=localhost:2181
kafkastore.topic=_schemas
debug=true
# Security specific params
kafkastore.bootstrap.servers=SASL_PLAINTEXT://localhost:9092
#kafkastore.bootstrap.servers=SASL_PLAINTEXT://or1010050208015:9092,SASL_PLAINTEXT://or1010050208016:9092,SASL_PLAINTEXT://or1010050208017:9092,SASL_PLAINTEXT://or1010050208018:9092
kafkastore.init.timeout.ms=10000
kafkastore.security.protocol=SASL_PLAINTEXT
zookeeper.set.acl=true
kafkastore.sasl.mechanism=PLAIN
kafkastore.group.id=schema-registry-consumer-group

I have followed instructions (modified for my use case and latest version
of confluent) from here:
https://www.confluent.io/blog/securing-confluent-schema-registry-apache-kafka/

When I try to start schema registry, it fails with following error:

[2017-09-06 06:05:23,790] ERROR Server died unexpectedly:
(io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain:52)
java.lang.IllegalArgumentException: Unable to subscribe to the Kafka topic
_schemas backing this data store. Topic may not exist.
    at
io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread.<init>(KafkaStoreReaderThread.java:130)


However, I have enabled schema registry user to produce/consume messages
for _schemas topic and authorizer debug messages show the same:

[2017-09-06 06:05:21,784] DEBUG operation = Read on resource =
Topic:_schemas from host = 127.0.0.1 is Allow based on acl = User:s
chemaregistry has Allow permission for operations: All from hosts: *
(kafka.authorizer.logger)
[2017-09-06 06:05:21,784] DEBUG Principal = User:schemaregistry is Allowed
Operation = Describe from host = 127.0.0.1 on resource
= Topic:_schemas (kafka.authorizer.logger)
[2017-09-06 06:05:22,788] DEBUG operation = Read on resource =
Topic:_schemas from host = 127.0.0.1 is Allow based on acl =
User:schemaregistry has Allow permission for operations: All from hosts: *
(kafka.authorizer.logger)
[2017-09-06 06:05:22,788] DEBUG Principal = User:schemaregistry is Allowed
Operation = Describe from host = 127.0.0.1 on resource = Topic:_schemas
(kafka.authorizer.logger)

Any ideas of what might be happening here?

Here're contents of my JAAS file that I use in the startup script:


[root@or1010050208015 kafka]# tail -1 /bin/schema-registry-start
exec $(dirname $0)/schema-registry-run-class ${EXTRA_ARGS}
-Djava.security.auth.login.config=/etc/schema-registry/schema_registry_client_jaas.properties
io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain "$@"


[root@or1010050208015 kafka]# cat
/etc/schema-registry/schema_registry_client_jaas.properties
/* Kafka Client */
KafkaClient {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="schemaregistry"
  password="schemaRegistry";
};

/* Zookeeper client */
Client {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    username="zk_admin"
    password="zk_admin_secret";
};


On Thu, Aug 31, 2017 at 1:35 PM, Shrikant Patel <SPatel@pdxinc.com> wrote:

> Yes, as per my understanding for now the only way to secure ZK is SASL.
> https://issues.apache.org/jira/browse/ZOOKEEPER-2125 once released ZK
> could also be secured using SSL.
>
> Also remember not use any OS user, anyone user on n\w who can connect to
> ZK host:port will be able to modify the ACLs.
>
> - Shri
>
> -----Original Message-----
> From: Manoj Murumkar [mailto:manoj.murumkar@gmail.com]
> Sent: Thursday, August 31, 2017 11:03 AM
> To: users@kafka.apache.org
> Subject: [EXTERNAL] Re: Using ACLs without Kerberos
>
> ***** Notice: This email was received from an external source *****
>
> >> Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
> Any OS user who got access to zookeeper can create ACLs for any Kafka
> principal.
>
> Thanks for that point. Appreciate it.
>
> On Thu, Aug 31, 2017 at 8:29 AM, Manikumar <manikumar.reddy@gmail.com>
> wrote:
>
> > There is no correlation between OS user and Kafka Principal/Username.
> > Here user name refers to the principal associated with the kafka
> > communication channel (Kerberos Principal, SASL/Plain username, Scram
> > username, SSL
> > certificate)
> >
> > Current kafka-acls.sh script directly contacts zookeeper to create ACLs.
> > Any OS user who got access to zookeeper can create ACLs for any Kafka
> > principal.
> > This can be restricted by enabling Zk Kerberos authentication.
> > http://kafka.apache.org/documentation/#zk_authz
> >
> > From Kafka 0.11.0.0, We can create ACLs using Java Admin API.
> >
> > On Thu, Aug 31, 2017 at 7:56 PM, Manoj Murumkar
> > <manoj.murumkar@gmail.com>
> > wrote:
> >
> > > Right, I am. Just to be clear, I am using kafka-acl script to
> > define/remove
> > > ACLs as a non-super user and it just works fine. I had expected it
> > > to
> > work
> > > only for super users and not for regular users ('nex37045' is a
> > > normal user).
> > >
> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
> > > `Topic:test`:
> > >      User:nex37045 has Deny permission for operations: Write from
> hosts:
> > *
> > >     User:skumarmu has Allow permission for operations: Write from
> hosts:
> > *
> > >
> > > [nex37045@or1010051029033 ~]$
> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > > zookeeper.connect=localhost:2181 --remove --allow-principal
> > User:skumarmu
> > > --operation Write --topic test
> > > Are you sure you want to remove ACLs:
> > >      User:skumarmu has Allow permission for operations: Write from
> > hosts: *
> > >  from resource `Topic:test`? (y/n)
> > > y
> > > Current ACLs for resource `Topic:test`:
> > >      User:nex37045 has Deny permission for operations: Write from
> hosts:
> > *
> > >
> > > [nex37045@or1010051029033 ~]$ kafka-acls --authorizer
> > > kafka.security.auth.SimpleAclAuthorizer --authorizer-properties
> > > zookeeper.connect=localhost:2181 --list Current ACLs for resource
> > > `Topic:test`:
> > >      User:nex37045 has Deny permission for operations: Write from
> hosts:
> > *
> > >
> > >
> > >
> > > On Thu, Aug 31, 2017 at 7:16 AM, Manikumar
> > > <manikumar.reddy@gmail.com>
> > > wrote:
> > >
> > > > Looks like you are already using SASL/PLAIN mechanism. Kafka
> > > > supports
> > > SASL
> > > > authentication framework.
> > > > KAFKA SASL supports GSSAPI (Kerberos), PLAIN or SCRAM mechanisms.
> > > > you
> > can
> > > > enable SSL encryption also
> > > >
> > > > http://kafka.apache.org/documentation.html#security
> > > >
> > > >
> > > > On Thu, Aug 31, 2017 at 7:28 PM, Manoj Murumkar <
> > > manoj.murumkar@gmail.com>
> > > > wrote:
> > > >
> > > > > Thanks Manikumar. I am testing the setup documented here:
> > > > > https://developer.ibm.com/opentech/2017/05/31/kafka-
> > acls-in-practice/
> > > > > (SASL_PLAINTEXT).
> > > > >
> > > > > I haven't setup any authentication for the tests. Thinking about
> > > > > it, authentication is a must have for authorization (so, kafka
> > > > > knows
> > who's
> > > > > making resource request), right?
> > > > >
> > > > > On Wed, Aug 30, 2017 at 11:39 PM, Manikumar <
> > manikumar.reddy@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Kafka default authorizer is used with secure authenticated
> > > > > > channels (SSL,SASL,SCRAM).
> > > > > > For plain text (non-secure) channels, the principal will be
> > > > > > always ANONYMOUS. Here you can authorize by ip-address.
> > > > > >
> > > > > > It's adviced to run on secure channels. you can try SASL/PLAIN
> > > > > > or
> > > SCRAM
> > > > > > mechanisms with/without SSL.
> > > > > > Pl, check Kafka docs for security considerations.
> > > > > >
> > > > > > >> I was testing if a non-admin principal (OS user) can
modify
> > > > > (add/remove)
> > > > > > ACLs and
> > > > > > it seems like it's possible.
> > > > > > How are you trying? Can you give more details about your setup?
> > What
> > > > > > authentication mechanism, etc.?
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > >
> > > > > > On Thu, Aug 31, 2017 at 2:49 AM, Manoj Murumkar <
> > > > > manoj.murumkar@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > We are evaluating how to put authorization in place for
> > > > > > > Kafka
> > > (around
> > > > > > > topics, mostly). Is it a good idea to do this without
> > > > > > > Kerberos? I
> > > was
> > > > > > > testing if a non-admin principal (OS user) can modify
> > (add/remove)
> > > > ACLs
> > > > > > and
> > > > > > > it seems like it's possible. If this is right behavior,
it's
> > > insecure
> > > > > and
> > > > > > > unusable. What do you guys think?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Manoj
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> This e-mail and its contents (to include attachments) are the property of
> National Health Systems, Inc., its subsidiaries and affiliates, including
> but not limited to Rx.com Community Healthcare Network, Inc. and its
> subsidiaries, and may contain confidential and proprietary or privileged
> information. If you are not the intended recipient of this e-mail, you are
> hereby notified that any unauthorized disclosure, copying, or distribution
> of this e-mail or of its attachments, or the taking of any unauthorized
> action based on information contained herein is strictly prohibited.
> Unauthorized use of information contained herein may subject you to civil
> and criminal prosecution and penalties. If you are not the intended
> recipient, please immediately notify the sender by telephone at
> 800-433-5719 or return e-mail and permanently delete the original e-mail.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message