kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From karan alang <karan.al...@gmail.com>
Subject Re: Kafka SSL error
Date Thu, 21 Sep 2017 00:00:47 GMT
Another point ..
on adding the chain.p12 (PKCS#12 format) .. order of the certs in the
keystore is as shown below

Alias : 1
Cert[1] -> server cert
Cert[2] -> Intermediate cert
Cert[3] -> Root cert

mentioning that, since one of the articles i read mentioned that the certs
imported should be in following order ->  1) Root 2) Intermediate 3) Server


On Wed, Sep 20, 2017 at 4:48 PM, karan alang <karan.alang@gmail.com> wrote:

> Hi Jakub,
>
> Thanks for the detailed note...
>
> here is the update ->
>
> I was able to convert the host.cert.pem to PKCS#12 & import the cert into
> the kafka.server.keystore.jks
> (also into kafka.server.truststore.jks)
> wrt the host.root.pem & host.intermed.pem certs - i'm assuming i need to
> convert them to PKCS#12 as well ?
> Or do i import them as PEM file ?
>
> I was not able to convert them to PKCS#12(since i understand the PKCS#12
> format requires a private key & the root.pem & intermed.pem donot have a
> private key associated)
> the error is shown below -
>
> openssl pkcs12 -export -out intermed.p12 -in hostname.issuer.pem -inkey
>> privatekey-kafka04-0920.key.pem -password pass:<password>
>> Enter pass phrase for privatekey-kafka04-0920.key.pem:
>> No certificate matches private key
>
>
>
> What i did next ->
>
> The host.chain.cert has all the 3 certs (server, intermediate, root)
> and i converted this to PKCS format & also was able to import into the
> keystore/truststore.
> I'm able to see the 3 certificates in the keystore/truststore under ONE
> alias.
>
>
> Now i get a different error shown below ->
>
> Command :
> openssl s_client -debug -connect <hostname>:9192 -tls1
>
> Error :
>
> depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>,
C =
>> US
>> verify error:num=20:unable to get local issuer certificate
>> verify return:1
>> depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>,
C =
>> US
>> verify error:num=27:certificate not trusted
>> verify return:1
>> depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>,
C =
>> US
>> verify error:num=21:unable to verify the first certificate
>> verify return:1
>> write to 0xd47830 [0xd98100] (12 bytes => 12 (0xC))
>> ---------
>> ---------
>> 0930 - 74 20 47 6c 6f 62 61 6c-20 43 41 0e               t Global CA.
>> 093f - <SPACES/NULS>
>> write to 0xd47830 [0xd98100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF))
>> 140456850737056:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:unexpected
>> message:s3_both.c:491:
>> --------
>> --------
>> SSL-Session:
>>     Protocol  : TLSv1
>> ------
>> ------
>>     Timeout   : 7200 (sec)
>>     Verify return code: 21 (unable to verify the first certificate)
>
>
>
> Any ideas on how to fix this ?
>
>
> On Wed, Sep 20, 2017 at 2:22 PM, Jakub Scholz <jakub@scholz.cz> wrote:
>
>> ad 1) The problem is that the signed certificate (host.cert.pem) which the
>> CA provides is only the public key. You have to combine it with the
>> private
>> key which you created when requesting the signed certificate. The private
>> key is never sent to the CA so they cannot provide it back. You or whoever
>> created the signing request should have it.
>>
>> ad 2) To create the keystore with your signed key, you have to take the
>> signed public key and the private key and create P12 file:
>> $ openssl pkcs12 -export -out server.p12 -in host.cert.pem -inkey host.key
>> -password pass:somepassword
>>
>> And afterwards convert it to keystore format:
>> $ keytool -importkeystore -srckeystore server.p12 -srcstoretype PKCS12
>> -srcstorepass somepassword -destkeystore server.keystore -deststoretype
>> JKS
>> -deststorepass somepassword -noprompt
>>
>> You can add the intermediate and root to the keystore as well. But when it
>> tells you that you already have them in the system wide keystore say "yes"
>> that you want to add them to the keystore again.
>>
>> ad 3) Do you want your clients to authenticate with a client certificates?
>> If not, you should not need the truststore for the broker. You only need a
>> truststore for the client which should contain the root and the
>> intermediate certificates. If you want clients to authenticate using
>> certificates you have to do basically the same as above for the client
>> (You
>> should use different certificate for cleints from the one in server - for
>> security reasons ;-)).
>>
>> ad 4) As I said, I think what you did before was that the broker basically
>> uses the self-signed cert you generated. This is probably the default
>> subject distinguished name of the self signed certificate. And since it is
>> self-signed openssl cannot verify its identity (unless you pass it the
>> public key of this exact self signed certificate).
>>
>> Jakub
>>
>> On Wed, Sep 20, 2017 at 10:46 PM, karan alang <karan.alang@gmail.com>
>> wrote:
>>
>> > Hello
>> > - thanks for the response
>> > Here is the update on the issue.
>> >
>> > I'm using certs signed/provided by org-wide CA (geotrust, not a
>> self-signed
>> > cert)
>> > The Signed(by the CA - geotrust) cert provided has 3 certificates
>> > - host.chain.pem (certificate chain - contains the Root, Intermediate,
>> > Signed Server cert )
>> > - host.intermediate.pem (intermediate certificate)
>> > - host.cert.pem (signed server sert)
>> > Steps :
>> > 1) Added the signed cert to kafka.server.keystore.jks
>> >
>> >
>> > >  keytool -import -alias servercert -trustcacerts -file host.cert.pem
>> > > -keystore kafka.server.keystore.jks
>> >
>> >
>> > *btw, Is the command above correct or needs to be changed ? *
>> >
>> > 2) Added the Intermediate & Signed cert to kafka.server.keystore.jks
>> >
>> > > keytool -import -alias intermediate -trustcacerts -file
>> host.issuer.pem
>> > > -keystore kafka.server.truststore.jks
>> > > keytool -import -alias servercert -trustcacerts -file host.cert.pem
>> > > -keystore kafka.server.truststore.jks
>> >
>> >
>> > When i try to add the Root Cert, it shows the following message.
>> >
>> > *Enter keystore password:  *
>> > * Certificate already exists in system-wide CA keystore under alias
>> > <geotrustglobalca [jdk]>*
>> > * Do you still want to add it to your own keystore? [no]:  *
>> >
>> > So, i assume the CA is already trusted .. do i still need to add this to
>> > the kafka.server.truststore.jks ?
>> > (I think i did that, and it did not fix the issue)
>> > 3) In the Broker side (server.properties file), the setting is as shown
>> > below
>> > (this is the same setting when i enabled SSL using OpenSSL, which is
>> > working fine btw)
>> >
>> > > ssl.keystore.location=/usr/hdp/2.5.3.0-37/confluent-3.2.
>> > > 2/kafkaSSL/kafka.server.keystore.jks
>> > > ssl.keystore.password=<passwd>
>> > > ssl.key.password=<passwd>
>> > > ssl.truststore.location=/usr/hdp/2.5.3.0-37/confluent-3.2.
>> > > 2/kafkaSSL/kafka.server.truststore.jks
>> > > ssl.truststore.password=<passwd>
>> > > ssl.client.auth=required
>> >
>> > 4) when i verify the SSL using command below
>> >
>> >  Command :
>> >
>> > >  openssl s_client -debug -connect <hostname>:9192 -tls1
>> >
>> >
>> >  Error snippet :
>> >
>> >
>> > >
>> > >
>> > > depth=0 C = US, ST = <State Name>, L = Unknown, O = <org name>.,
OU =
>> > > <OU>, CN = <localhost>
>> > >
>> > verify error:num=18:self signed certificate
>> > >
>> > > verify return:1
>> > >  ....................
>> > >
>> >              -----------------
>> >              ------------------
>> >
>> >
>> > >  063d - <SPACES/NULS>
>> > > write to 0x860830 [0x8b1100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF))
>> > > 140487373043616:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:unex
>> pected
>> > > message:s3_both.c:491:
>> >
>> >
>> > Any ideas on this ?
>> >         What needs to be done to debug or fix this ?
>> >
>> > On Wed, Sep 20, 2017 at 12:33 AM, Jakub Scholz <jakub@scholz.cz> wrote:
>> >
>> > > Hi,
>> > >
>> > > Looking at your commands it looks as if you generated a self signed
>> key
>> > for
>> > > server, self signed key for client and then imported the CA keys
>> public
>> > > keys into the truststores. I don’t think this will work because now
>> you
>> > > have two different self signed keys in the keystores and the
>> presumably
>> > the
>> > > CA public key in your truststores.
>> > >
>> > > Does the cert.pem actualy contain the private key? In case it doesn’t
>> > > contain the private key you need to get the private key first. If yes,
>> > you
>> > > have to convert it into the keystore format using this guide:
>> > > https://docs.oracle.com/cd/E35976_01/server.740/es_admin/
>> > > src/tadm_ssl_convert_pem_to_jks.html
>> > > You basically have to first use OpenSSL to create PKCS12 key and that
>> can
>> > > be converted into keystore.
>> > >
>> > > BTW: If you run your application with system property
>> “javax.net.debug”
>> > set
>> > > to “ssl” it will generate a lot of useful debug information which will
>> > help
>> > > to understand what is going on and fix this.
>> > >
>> > > Jakub
>> > >
>> > > On Tue, 19 Sep 2017 at 23:44, karan alang <karan.alang@gmail.com>
>> wrote:
>> > >
>> > > > Hello All -
>> > > > I was able to set up SSL for the Kafka brokers, using OpenSSL.
>> > > >
>> > > > however, I'm having issues with setting up SSL using the pem file
>> (i.e.
>> > > SSL
>> > > > certificate - certified by CA, provided by the company)
>> > > >
>> > > > Here is what i've done -
>> > > > created the server/client keystore & truststore files and imported
>> the
>> > > > provided cert.pem file
>> > > >
>> > > > keytool -keystore kafka.server.keystore.jks -alias localhost
>> -validity
>> > > 365
>> > > > -genkey
>> > > > keytool -keystore kafka.server.truststore.jks -alias CARoot -import
>> > -file
>> > > > cert.pem
>> > > > keytool -keystore kafka.client.truststore.jks -alias CARoot -import
>> > -file
>> > > > cert.pem
>> > > > keytool -keystore kafka.server.keystore.jks -alias CARoot -import
>> -file
>> > > > cert.pem
>> > > > keytool -keystore kafka.client.keystore.jks -alias localhost
>> -validity
>> > > 365
>> > > > -genkey
>> > > > keytool -keystore kafka.client.keystore.jks -alias CARoot -import
>> -file
>> > > > cert.pem
>> > > >
>> > > > I've a console producer pushing data in to the topic, and gives
>> error
>> > as
>> > > > shown below ->
>> > > >
>> > > >
>> > > > Caused by: javax.net.ssl.SSLProtocolException: Handshake message
>> > > sequence
>> > > > > violation, state = 1, type = 1
>> > > > > at
>> > > > >
>> > > > sun.security.ssl.ServerHandshaker.processMessage(
>> > > ServerHandshaker.java:213)
>> > > > > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
>> > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:966)
>> > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:963)
>> > > > > at java.security.AccessController.doPrivileged(Native Method)
>> > > > > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.
>> > java:1416)
>> > > > > at
>> > > > >
>> > > > org.apache.kafka.common.network.SslTransportLayer.runDelegat
>> edTasks(
>> > > SslTransportLayer.java:336)
>> > > > > at
>> > > > >
>> > > > org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(
>> > > SslTransportLayer.java:417)
>> > > > > ... 7 more
>> > > >
>> > > >
>> > > >
>> > > > Any ideas on what the issue might be ?
>> > > > thanks for help in advance!
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message