nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Pablo Gardella <gardellajuanpa...@gmail.com>
Subject Re: ElasticSearchClientServiceImpl not working for secured ElasticSearch
Date Fri, 18 Oct 2019 17:10:17 GMT
Any ETA for Nifi 1.10 release?

On Fri, 18 Oct 2019 at 13:39, Mike Thomsen <mikerthomsen@gmail.com> wrote:

> Peter,
>
> Are you configuring the service as a trust-only configuration? If so,
> that's been addressed in the 1.10 which is due for release in the near(ish)
> future.
>
> https://issues.apache.org/jira/browse/NIFI-6228
>
> Thanks,
>
> Mike
>
> On Fri, Oct 18, 2019 at 11:06 AM Peter Moberg <peter.moberg@gmail.com>
> wrote:
>
>> As a follow-up.
>>
>> On the Nifi node I am able to do a GET to Elastic Search using curl. I
>> specify the —cacert option giving it the self-signed root certificate.
>>
>> Of course, this isn’t using the TrustStore but I am able to use the
>> TrustStore if I use other ES processors… just not the
>> ElasticSearchClientServicesImpl.
>>
>> On Oct 18, 2019, 12:48 AM -0500, Peter Moberg <peter.moberg@gmail.com>,
>> wrote:
>>
>> Hi Andy,
>>
>> thanks for your suggestions. Here is what I have tried so far (still no
>> luck).
>>
>> Connecting with openssl and viewing the certs it presents
>>
>> *openssl s_client -connect quickstart-es-http.es
>> <http://quickstart-es-http.es>-cluster -showcerts*
>>
>> If I then look inside the server cert I can find this
>>
>>
>>
>>
>>
>> *Server Cert: Issuer: OU = quickstart, CN = quickstart-http X509v3
>> Subject Alternative Name: DNS:quickstart-es-http.es
>> <http://quickstart-es-http.es>-cluster.es.local, DNS:quickstart-es-http,
>> DNS:quickstart-es-http.es <http://quickstart-es-http.es>-cluster.svc,
>> DNS:quickstart-es-http.es <http://quickstart-es-http.es>-cluster*
>>
>>
>> If I look in to the self-signed root cert I find this:
>>
>>
>> *Root Cert: Subject: OU = quickstart, CN = quickstart-http*
>>
>>
>> I now double check  my trust store to make sure the Root Cert is there.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Trust store content Your keystore contains 1 entry Alias name:
>> ca_elastic Creation date: Oct 16, 2019 Entry type: trustedCertEntry Owner:
>> CN=quickstart-http, OU=quickstart Issuer: CN=quickstart-http, OU=quickstart
>> Serial number: 5aa50b6806d2394fff6f98d2b7d4c287 Valid from: Fri Oct 11
>> 14:35:01 UTC 2019 until: Sat Oct 10 14:36:01 UTC 2020 Certificate
>> fingerprints: MD5: 1E:E3:33:13:EA:AC:B5:61:23:DE:2E:1A:D7:9C:AA:F0 SHA1:
>> 62:EC:5B:EB:32:6A:38:3D:6A:6B:F7:10:5A:DE:E6:F1:F0:5B:07:99 SHA256:
>> B4:B5:06:9C:50:5F:E8:A1:58:7C:C7:2C:37:52:2F:E0:CF:32:18:18:68:E4:C7:37:F8:82:B3:BC:61:EB:5B:CF
>> Signature algorithm name: SHA256withRSA Subject Public Key Algorithm:
>> 2048-bit RSA key Version: 3 Extensions: #1: ObjectId: 2.5.29.19
>> Criticality=true BasicConstraints:[ CA:true PathLen:2147483647 ] #2:
>> ObjectId: 2.5.29.37 Criticality=false ExtendedKeyUsages [ serverAuth
>> clientAuth ] #3: ObjectId: 2.5.29.15 Criticality=true KeyUsage [
>> DigitalSignature Key_CertSign ]*
>>
>> So everything looks Ok. But when I run the
>> ElasticSearchClientServicesImpl with a SSLContext pointing to my trust
>> store I still get the following output in the log.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine
>> problem at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at
>> sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1728) at
>> sun.security.ssl.Handshaker.fatalSE(Handshaker.java:330) at
>> sun.security.ssl.Handshaker.fatalSE(Handshaker.java:322) at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1633)
>> at
>> sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
>> at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052) at
>> sun.security.ssl.Handshaker$1.run(Handshaker.java:992) at
>> sun.security.ssl.Handshaker$1.run(Handshaker.java:989) at
>> java.security.AccessController.doPrivileged(Native Method) at
>> sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1467) at
>> org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283)
>> at
>> org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353)
>> ... 9 common frames omitted Caused by:
>> sun.security.validator.ValidatorException: PKIX path building failed:
>> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
>> valid certification path to requested target at
>> sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397) at
>> sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
>> at sun.security.validator.Validator.validate(Validator.java:262) at
>> sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)
>> at
>> sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
>> at
>> sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1620)
>> ... 17 common frames omitted*
>>
>> Both the Nifi install and Elastic Search install is running in
>> Kubernetes. The address I am using is a service address that is backed by 3
>> ES instances. However, I double checked all three of the ES nodes to make
>> sure that they returned back the same SSL cert and they did.
>>
>> The only thing I haven't been able to figure out is how to check if
>> Kubernetes/ES reacts differently when you do a GET vs POST. Feels strange
>> that it would return different SSL certs but stranger things have happened…
>>
>>
>>
>> On Oct 17, 2019, 3:25 PM -0500, Andy LoPresto <alopresto@apache.org>,
>> wrote:
>>
>> Hi Peter,
>>
>> If you can use openssl’s s_client command (example below) to connect to
>> the endpoint and verify that the hostname matches the certificate and that
>> the certificate contains a SubjectAlternativeName entry with that hostname
>> (see RFC 6125 [1] for more details), this should help you debug the issue.
>> The cause of the PKIX error is that the truststore doesn’t contain a
>> certificate (or certificate chain) which matches the hostname presented by
>> the remote endpoint. I think you understand that based on your message. The
>> underlying reason for this is could be one of the following:
>>
>> * the server is behind an interface which responds differently to GET and
>> POST/PUT requests
>> * there is a load-balancer which is directing the requests coincidentally
>> to different backend servers (one has the right cert; the other doesn’t)
>> * I recall something around the addition of (some) Elastic Search
>> components which handled TLS in an ES client-specific manner; I remember
>> advocating for standard NiFi TLS interaction here but I am not sure what
>> was ultimately contributed. If it’s not one of the above issues, I can
>> investigate further.
>>
>> Hopefully this helps.
>>
>> [1] https://tools.ietf.org/html/rfc6125#section-6.4.4
>>
>> s_client example:
>>
>> $ openssl s_client -connect <host:port> -debug -state -cert
>> <path_to_your_cert.pem> -key <path_to_your_key.pem> -CAfile
>> <path_to_your_CA_cert.pem>
>>
>> Andy LoPresto
>> alopresto@apache.org
>> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> On Oct 16, 2019, at 8:37 PM, Peter Moberg <peter.moberg@gmail.com> wrote:
>>
>> I have an Elastic Search cluster that is setup with SSL. It uses a
>> self-signed cert for this. I am working with Apache Nifi 1.9.2. I have a
>> flow that has the PutElasticSearchHttp component. I have setup a
>> SSLContextService for that component where I have specified a trust store
>> that has the self-signed cert from ES. I specify an https endpoint to
>> access Elastic Search and Im having no issues populating my Elastic Search
>> instance using this flow.
>>
>> I have another flow where I want to do some lookups. So I have been using
>> the LookupRecord processor. That one I have associated with an
>> ElasticSearchClientServiceImpl which I have setup to  point to the same
>> SSLContextService as used above. I specified the same HTTPS Url (triple
>> checked this). However, when I run this second Flow I am not able to verify
>> the ES server's self-signed certificate.
>>
>> I check the nifi-app.log and it says:
>> Caused by: sun.security.provider.certpath.SunCertPathBuilderException:
>> unable to find valid certification path to requested target
>>
>> I am a bit surprised that I am not able to verify the same server
>> certificate in the two different flows.
>>
>> Completely stuck on this so if anyone have any pointers please let me
>> know.
>>
>> Thanks,
>>
>> Peter
>>
>>
>>

Mime
View raw message