kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From karan alang <karan.al...@gmail.com>
Subject Re: Kafka SSL error
Date Wed, 20 Sep 2017 23:48:20 GMT
Hi Jakub,

Thanks for the detailed note...

here is the update ->

I was able to convert the host.cert.pem to PKCS#12 & import the cert into
the kafka.server.keystore.jks
(also into kafka.server.truststore.jks)
wrt the host.root.pem & host.intermed.pem certs - i'm assuming i need to
convert them to PKCS#12 as well ?
Or do i import them as PEM file ?

I was not able to convert them to PKCS#12(since i understand the PKCS#12
format requires a private key & the root.pem & intermed.pem donot have a
private key associated)
the error is shown below -

openssl pkcs12 -export -out intermed.p12 -in hostname.issuer.pem -inkey
> privatekey-kafka04-0920.key.pem -password pass:<password>
> Enter pass phrase for privatekey-kafka04-0920.key.pem:
> No certificate matches private key



What i did next ->

The host.chain.cert has all the 3 certs (server, intermediate, root)
and i converted this to PKCS format & also was able to import into the
keystore/truststore.
I'm able to see the 3 certificates in the keystore/truststore under ONE
alias.


Now i get a different error shown below ->

Command :
openssl s_client -debug -connect <hostname>:9192 -tls1

Error :

depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>,
C = US
> verify error:num=20:unable to get local issuer certificate
> verify return:1
> depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>,
C =
> US
> verify error:num=27:certificate not trusted
> verify return:1
> depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>,
C =
> US
> verify error:num=21:unable to verify the first certificate
> verify return:1
> write to 0xd47830 [0xd98100] (12 bytes => 12 (0xC))
> ---------
> ---------
> 0930 - 74 20 47 6c 6f 62 61 6c-20 43 41 0e               t Global CA.
> 093f - <SPACES/NULS>
> write to 0xd47830 [0xd98100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF))
> 140456850737056:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:unexpected
> message:s3_both.c:491:
> --------
> --------
> SSL-Session:
>     Protocol  : TLSv1
> ------
> ------
>     Timeout   : 7200 (sec)
>     Verify return code: 21 (unable to verify the first certificate)



Any ideas on how to fix this ?


On Wed, Sep 20, 2017 at 2:22 PM, Jakub Scholz <jakub@scholz.cz> wrote:

> ad 1) The problem is that the signed certificate (host.cert.pem) which the
> CA provides is only the public key. You have to combine it with the private
> key which you created when requesting the signed certificate. The private
> key is never sent to the CA so they cannot provide it back. You or whoever
> created the signing request should have it.
>
> ad 2) To create the keystore with your signed key, you have to take the
> signed public key and the private key and create P12 file:
> $ openssl pkcs12 -export -out server.p12 -in host.cert.pem -inkey host.key
> -password pass:somepassword
>
> And afterwards convert it to keystore format:
> $ keytool -importkeystore -srckeystore server.p12 -srcstoretype PKCS12
> -srcstorepass somepassword -destkeystore server.keystore -deststoretype JKS
> -deststorepass somepassword -noprompt
>
> You can add the intermediate and root to the keystore as well. But when it
> tells you that you already have them in the system wide keystore say "yes"
> that you want to add them to the keystore again.
>
> ad 3) Do you want your clients to authenticate with a client certificates?
> If not, you should not need the truststore for the broker. You only need a
> truststore for the client which should contain the root and the
> intermediate certificates. If you want clients to authenticate using
> certificates you have to do basically the same as above for the client (You
> should use different certificate for cleints from the one in server - for
> security reasons ;-)).
>
> ad 4) As I said, I think what you did before was that the broker basically
> uses the self-signed cert you generated. This is probably the default
> subject distinguished name of the self signed certificate. And since it is
> self-signed openssl cannot verify its identity (unless you pass it the
> public key of this exact self signed certificate).
>
> Jakub
>
> On Wed, Sep 20, 2017 at 10:46 PM, karan alang <karan.alang@gmail.com>
> wrote:
>
> > Hello
> > - thanks for the response
> > Here is the update on the issue.
> >
> > I'm using certs signed/provided by org-wide CA (geotrust, not a
> self-signed
> > cert)
> > The Signed(by the CA - geotrust) cert provided has 3 certificates
> > - host.chain.pem (certificate chain - contains the Root, Intermediate,
> > Signed Server cert )
> > - host.intermediate.pem (intermediate certificate)
> > - host.cert.pem (signed server sert)
> > Steps :
> > 1) Added the signed cert to kafka.server.keystore.jks
> >
> >
> > >  keytool -import -alias servercert -trustcacerts -file host.cert.pem
> > > -keystore kafka.server.keystore.jks
> >
> >
> > *btw, Is the command above correct or needs to be changed ? *
> >
> > 2) Added the Intermediate & Signed cert to kafka.server.keystore.jks
> >
> > > keytool -import -alias intermediate -trustcacerts -file host.issuer.pem
> > > -keystore kafka.server.truststore.jks
> > > keytool -import -alias servercert -trustcacerts -file host.cert.pem
> > > -keystore kafka.server.truststore.jks
> >
> >
> > When i try to add the Root Cert, it shows the following message.
> >
> > *Enter keystore password:  *
> > * Certificate already exists in system-wide CA keystore under alias
> > <geotrustglobalca [jdk]>*
> > * Do you still want to add it to your own keystore? [no]:  *
> >
> > So, i assume the CA is already trusted .. do i still need to add this to
> > the kafka.server.truststore.jks ?
> > (I think i did that, and it did not fix the issue)
> > 3) In the Broker side (server.properties file), the setting is as shown
> > below
> > (this is the same setting when i enabled SSL using OpenSSL, which is
> > working fine btw)
> >
> > > ssl.keystore.location=/usr/hdp/2.5.3.0-37/confluent-3.2.
> > > 2/kafkaSSL/kafka.server.keystore.jks
> > > ssl.keystore.password=<passwd>
> > > ssl.key.password=<passwd>
> > > ssl.truststore.location=/usr/hdp/2.5.3.0-37/confluent-3.2.
> > > 2/kafkaSSL/kafka.server.truststore.jks
> > > ssl.truststore.password=<passwd>
> > > ssl.client.auth=required
> >
> > 4) when i verify the SSL using command below
> >
> >  Command :
> >
> > >  openssl s_client -debug -connect <hostname>:9192 -tls1
> >
> >
> >  Error snippet :
> >
> >
> > >
> > >
> > > depth=0 C = US, ST = <State Name>, L = Unknown, O = <org name>.,
OU =
> > > <OU>, CN = <localhost>
> > >
> > verify error:num=18:self signed certificate
> > >
> > > verify return:1
> > >  ....................
> > >
> >              -----------------
> >              ------------------
> >
> >
> > >  063d - <SPACES/NULS>
> > > write to 0x860830 [0x8b1100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF))
> > > 140487373043616:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:
> unexpected
> > > message:s3_both.c:491:
> >
> >
> > Any ideas on this ?
> >         What needs to be done to debug or fix this ?
> >
> > On Wed, Sep 20, 2017 at 12:33 AM, Jakub Scholz <jakub@scholz.cz> wrote:
> >
> > > Hi,
> > >
> > > Looking at your commands it looks as if you generated a self signed key
> > for
> > > server, self signed key for client and then imported the CA keys public
> > > keys into the truststores. I don’t think this will work because now you
> > > have two different self signed keys in the keystores and the presumably
> > the
> > > CA public key in your truststores.
> > >
> > > Does the cert.pem actualy contain the private key? In case it doesn’t
> > > contain the private key you need to get the private key first. If yes,
> > you
> > > have to convert it into the keystore format using this guide:
> > > https://docs.oracle.com/cd/E35976_01/server.740/es_admin/
> > > src/tadm_ssl_convert_pem_to_jks.html
> > > You basically have to first use OpenSSL to create PKCS12 key and that
> can
> > > be converted into keystore.
> > >
> > > BTW: If you run your application with system property “javax.net.debug”
> > set
> > > to “ssl” it will generate a lot of useful debug information which will
> > help
> > > to understand what is going on and fix this.
> > >
> > > Jakub
> > >
> > > On Tue, 19 Sep 2017 at 23:44, karan alang <karan.alang@gmail.com>
> wrote:
> > >
> > > > Hello All -
> > > > I was able to set up SSL for the Kafka brokers, using OpenSSL.
> > > >
> > > > however, I'm having issues with setting up SSL using the pem file
> (i.e.
> > > SSL
> > > > certificate - certified by CA, provided by the company)
> > > >
> > > > Here is what i've done -
> > > > created the server/client keystore & truststore files and imported
> the
> > > > provided cert.pem file
> > > >
> > > > keytool -keystore kafka.server.keystore.jks -alias localhost
> -validity
> > > 365
> > > > -genkey
> > > > keytool -keystore kafka.server.truststore.jks -alias CARoot -import
> > -file
> > > > cert.pem
> > > > keytool -keystore kafka.client.truststore.jks -alias CARoot -import
> > -file
> > > > cert.pem
> > > > keytool -keystore kafka.server.keystore.jks -alias CARoot -import
> -file
> > > > cert.pem
> > > > keytool -keystore kafka.client.keystore.jks -alias localhost
> -validity
> > > 365
> > > > -genkey
> > > > keytool -keystore kafka.client.keystore.jks -alias CARoot -import
> -file
> > > > cert.pem
> > > >
> > > > I've a console producer pushing data in to the topic, and gives error
> > as
> > > > shown below ->
> > > >
> > > >
> > > > Caused by: javax.net.ssl.SSLProtocolException: Handshake message
> > > sequence
> > > > > violation, state = 1, type = 1
> > > > > at
> > > > >
> > > > sun.security.ssl.ServerHandshaker.processMessage(
> > > ServerHandshaker.java:213)
> > > > > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
> > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:966)
> > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:963)
> > > > > at java.security.AccessController.doPrivileged(Native Method)
> > > > > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.
> > java:1416)
> > > > > at
> > > > >
> > > > org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(
> > > SslTransportLayer.java:336)
> > > > > at
> > > > >
> > > > org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(
> > > SslTransportLayer.java:417)
> > > > > ... 7 more
> > > >
> > > >
> > > >
> > > > Any ideas on what the issue might be ?
> > > > thanks for help in advance!
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message