nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: Communication Error Between NiFi and Registry: Error retrieving all buckets
Date Wed, 07 Aug 2019 18:00:56 GMT
Hard to say why it's not working, but it should be setup in the following way...

In registry, there needs to be two users...

1) nmaynes@EXAMPLE.COM

This user needs read access to at least one bucket, or needs read to
all buckets in the special privileges.

2) CN=nifi.example.com, L=Anytown, ST=IN, C=US

This user needs the special privileges for Proxy and Read all buckets.

When you are in NiFi as nmaynes@EXAMPLE.COM  and perform an action
that talks to registry, it makes a request using NiFi's server cert
(user 2 above) and sends the end user as a proxied entity (user 1
above). When the request hits registry it checks those two users
against the permissions described above.

If you are getting a 401 then either one of the policies is missing,
or one of the identities is not lining up.


On Wed, Aug 7, 2019 at 11:09 AM Nathan Maynes <nathanmaynes@gmail.com> wrote:
>
> I tried with and without the quotes. I am getting the same error. Do I need to be restarting
NiFi or using a clean browser session each time I make an update? My intuition says no since
the authentication should be happening server side but wondering if I am missing something.
>
> On Wed, Aug 7, 2019 at 10:13 AM Bryan Bende <bbende@gmail.com> wrote:
>>
>> I don't think there should be quotes around the NiFi identity...
>>
>> You have:
>>
>> identity="&quot;CN=nifi.example.com, L=Anytown, ST=IN, C=US&quot;"
>>
>> It should be:
>>
>> identity="CN=nifi.example.com, L=Anytown, ST=IN, C=US"
>>
>> On Wed, Aug 7, 2019 at 10:02 AM Nathan Maynes <nathanmaynes@gmail.com> wrote:
>> >
>> > Thanks for that information Nathan. I went ahead and updated the Nifi-Registry
user to have the name  "CN=nifi.example.com, L=Anytown, ST=IN, C=US", which was copied out
of the certificate that NiFi is using as its keystore, as defined in nifi.properties ->
nifi.security.keystore. The error persisted. For good measure, I went ahead and restarted
the Registry. I then checked the users.xml file and found that the user string had been added.
There is an entry for me, and one for the DN string I expect from NiFi. See sanitized example
below.
>> >
>> > <users>
>> >         <user identifier="guid-1" identity="nmaynes@EXAMPLE.COM"/>
>> >         <user identifier="guid-2" identity="&quot;CN=nifi.example.com,
L=Anytown, ST=IN, C=US&quot;"/>
>> >     </users>
>> >
>> > I checked the nifi-registry-app.log to see if it contained extra information.
Here is what I found,
>> >
>> > INFO [NiFi Registry Web Server-14] o.a.n.r.w.s.NiFiRegistrySecurityConfig Client
could not be authenticated due to: org.springframework.security.authentication.AuthenticationCredentialsNotFoundException:
An Authentication object was not found in the SecurityContext Returning 401 response.
>> >
>> > I am not sure what to check at this point.
>> >
>> > On Tue, Aug 6, 2019 at 3:11 PM Nathan Gough <thenatog@gmail.com> wrote:
>> >>
>> >> Nathan,
>> >>
>> >> You would need to create a user in NiFi registry with the exact DN of the
NiFi certificate being used to access NiFi registry.
>> >>
>> >> From your example, you would create a user in NiFi registry with the exact
string "CN=nifi.example.com, L=Anytown, ST=IN, C=US" and apply the read buckets and proxy
user permissions.
>> >>
>> >> Cheers,
>> >> Nathan
>> >>
>> >> On Tue, Aug 6, 2019 at 2:22 PM Nathan Maynes <nathanmaynes@gmail.com>
wrote:
>> >>>
>> >>> Thanks for pointing this out Bryan. To be sure I was entering the information
correctly I used the Java Keytool to examine the certificate contents. Here is what the sanitized
output looks like.
>> >>>
>> >>> $ keytool -list -v -keystore nifi.jks
>> >>>
>> >>> Keystore type: jks
>> >>> Keystore provider: SUN
>> >>>
>> >>> Your keystore contains 1 entry
>> >>>
>> >>> Alias name: nifi-https
>> >>> Creation date: Jun 20, 2019
>> >>> Entry type: PrivateKeyEntry
>> >>> Certificate chain length: 3
>> >>> Certificate[1]:
>> >>> Owner: CN=nifi.example.com, L=Anytown, ST=IN, C=US
>> >>> Issuer: CN=Internal Intermediate CA (2015), DC=EXAMPLE, DC=com
>> >>> Serial number: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>> >>> Valid from: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>> >>>
>> >>> ...[two more certs in chain]
>> >>>
>> >>>
>> >>> The user I create for the registry has the following value:
>> >>>
>> >>> "CN=nifi.example.com, OU=NIFI"
>> >>>
>> >>> I then granted that user permission to read buckets and proxy user requests.
I am not sure the organizational unit, OU in the example above, is NIFI. I have created a
number of other users with slight variations on the CN and OU values but any attempt to connect
the two services fails. Still getting the error, "Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException:
Error retrieving all buckets: An Authentication object was not found in the SecurityContext
Contact the system administrator."
>> >>>
>> >>> I am trying to use the certificate issued to the NiFi server. Do I need
to create a unique certificate for authentication between the two services?
>> >>>
>> >>> On Mon, Aug 5, 2019 at 3:12 PM Bryan Bende <bbende@gmail.com>
wrote:
>> >>>>
>> >>>> Your NiFi identity will always be the DN of the server certificate
>> >>>> that NiFi is using which is specified in nifi.security.keystore
in
>> >>>> nifi.properties.
>> >>>>
>> >>>> Kerberos is only for the end-users that use the NiFi web application.
>> >>>>
>> >>>> In the video around 6:45 where a user is added to registry like
>> >>>> "CN=localhost, OU=NIFI", you would do the same thing, except it
would
>> >>>> be the value coming from your NiFi server cert, so it would have
your
>> >>>> hostname and possibly a different OU.
>> >>>>
>> >>>> On Mon, Aug 5, 2019 at 2:57 PM Nathan Maynes <nathanmaynes@gmail.com>
wrote:
>> >>>> >
>> >>>> > The video shows appears to show certificate based access. When
I set the NiFi Identity 1 for a Kerberos scheme should it follow the user@DOMAIN.COM format?
If it does, would the NiFi Identity 1 for localhost be nifi@LOCALHOST?
>> >>>> >
>> >>>> > On Mon, Aug 5, 2019 at 1:47 PM Bryan Bende <bbende@gmail.com>
wrote:
>> >>>> >>
>> >>>> >> Hello,
>> >>>> >>
>> >>>> >> I believe the video should cover this, but did you add
a user
>> >>>> >> representing your NiFi instance and grant it the permissions
for proxy
>> >>>> >> and read all buckets?
>> >>>> >>
>> >>>> >> That is what "NiFi Identity 1" would have done, but that
only gets
>> >>>> >> used on initial setup, so you would do it from the UI now.
>> >>>> >>
>> >>>> >> -Bryan
>> >>>> >>
>> >>>> >> On Mon, Aug 5, 2019 at 1:30 PM Nathan Maynes <nathanmaynes@gmail.com>
wrote:
>> >>>> >> >
>> >>>> >> > Hopefully I can get some guidance on configuring secure
communication between NiFi and NiFi-Registry. The Error I have been trying to resolve occurs
when trying to send a processor group to NiFi-Registry for versioning. Below is the error
message displayed in the NiFi UI.
>> >>>> >> >
>> >>>> >> > "Unable to obtain listing of buckets: org.apache.nifi.registry.client.NiFiRegistryException:
Error retrieving all buckets: An Authentication object was not found in the SecurityContext
Contact the system administrator. "
>> >>>> >> >
>> >>>> >> > I started out by watching the tutorial video "Setting
Up a Secure NiFi to Integrate with a Secure NiFi Registry" posted on the Registry home page.
I am using a Kerberos file-based authentication scheme with the initial admin and initial
user set to the same value, eg "name@DOMAIN.COM." (This is a sanitized value and is used in
the configuration example below) It is based on the configuration we are using for NiFi. My
nifi-registry.properties file has the following relevant values set.
>> >>>> >> >
>> >>>> >> > # security properties #
>> >>>> >> > nifi.registry.security.keystore=/etc/ssl/nifi2019.p12
>> >>>> >> > nifi.registry.security.keystoreType=pkcs12
>> >>>> >> > nifi.registry.security.keystorePasswd=XXXXXX
>> >>>> >> > nifi.registry.security.keyPasswd=XXXXXX
>> >>>> >> > nifi.registry.security.truststore=/path/to/cacerts
>> >>>> >> > nifi.registry.security.truststoreType=jks
>> >>>> >> > nifi.registry.security.truststorePasswd=XXXXXX
>> >>>> >> > nifi.registry.security.needClientAuth=false
>> >>>> >> > nifi.registry.security.authorizers.configuration.file=./conf/authorizers.xml
>> >>>> >> > nifi.registry.security.authorizer=managed-authorizer
>> >>>> >> > nifi.registry.security.identity.providers.configuration.file=./conf/identity-providers.xml
>> >>>> >> > nifi.registry.security.identity.provider=kerberos-identity-provider
>> >>>> >> >
>> >>>> >> > ...
>> >>>> >> >
>> >>>> >> > # kerberos properties #
>> >>>> >> > nifi.registry.kerberos.krb5.file=/etc/krb5.conf
>> >>>> >> > nifi.registry.kerberos.spnego.principal=svcnififsaccess/DOMAIN.COM
>> >>>> >> > nifi.registry.kerberos.spnego.keytab.location=/etc/ssl/example.keytab
>> >>>> >> > nifi.registry.kerberos.spnego.authentication.expiration=2
hours
>> >>>> >> >
>> >>>> >> > And in authorizers.xml I have:
>> >>>> >> >
>> >>>> >> > <userGroupProvider>
>> >>>> >> >         <identifier>file-user-group-provider</identifier>
>> >>>> >> >         <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
>> >>>> >> >         <property name="Users File">./conf/users.xml</property>
>> >>>> >> >         <property name="Initial User Identity 1">user@DOMAIN.COM</property>
>> >>>> >> > </userGroupProvider>
>> >>>> >> >
>> >>>> >> > <accessPolicyProvider>
>> >>>> >> >         <identifier>file-access-policy-provider</identifier>
>> >>>> >> >         <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
>> >>>> >> >         <property name="User Group Provider">file-user-group-provider</property>
>> >>>> >> >         <property name="Authorizations File">./conf/authorizations.xml</property>
>> >>>> >> >         <property name="Initial Admin Identity">user@DOMAIN.COM</property>
>> >>>> >> >         <property name="NiFi Identity 1"></property>
>> >>>> >> > </accessPolicyProvider>
>> >>>> >> > <authorizer>
>> >>>> >> >         <identifier>managed-authorizer</identifier>
>> >>>> >> >         <class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class>
>> >>>> >> >         <property name="Access Policy Provider">file-access-policy-provider</property>
>> >>>> >> > </authorizer>
>> >>>> >> >
>> >>>> >> > The SSL configuration appears to be correctly set.
I am able to access via username and password, the NiFi Registry UI. Despite my best efforts
to read the documentation, I am unclear on the following points.
>> >>>> >> >
>> >>>> >> > Do I need to set the <property name="NiFi Identity
1"></property>?
>> >>>> >> > Is there any special considerations I need to be aware
of if I run NiFi and the NiFi Registry from the same box and use the same domain name?
>> >>>> >> >
>> >>>> >> > Any guidance you may be able to share would be appreciated.
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > --
>> >>>> >> > Nathan Maynes
>> >>>> >> > @nathanmaynes
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > --
>> >>>> > Nathan Maynes
>> >>>> > @nathanmaynes
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Nathan Maynes
>> >>> @nathanmaynes
>> >
>> >
>> >
>> > --
>> > Nathan Maynes
>> > @nathanmaynes
>
>
>
> --
> Nathan Maynes
> @nathanmaynes

Mime
View raw message