trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Susan Hinrichs <shinr...@verizonmedia.com>
Subject Re: Client / TS certificates 7.x vs 8.x
Date Wed, 06 Jan 2021 22:39:18 GMT
Looking at our production environment, I see that we use an absolute path
in proxy.config.ssl.CA.cert.filename.  Could you try setting that to
/ssl/ca.pem?  Perhaps a bug came in with the path manipulation.

On Wed, Jan 6, 2021 at 3:01 PM Zack Bartel <zbartel@proofpoint.com> wrote:

> Thanks for the help!
> Yes, the client certificate is directly signed by the certificate in
> ca.pem. There is no intermediary. I did not know about the "ssl" debug flag
> but this is very helpful. It seems your theory about failing the client
> cert verification was correct.
>
> [Jan  6 08:21:22.635] {0x7ff241b10700} DEBUG: <SSLNetVConnection.cc:1213
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslnetvconnection.cc-3A1213&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=NeldsD4eLgkhoyTV-xMLLQXlHwznW8HxcVoSqwV9Xj0&e=>
> (sslServerHandShakeEvent)> (ssl-diag) SSL handshake error:
> SSL_ERROR_WANT_READ (2), errno=0
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLNetVConnection.cc:1139
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslnetvconnection.cc-3A1139&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=2e8RrV4SJTDNVqVPg9Jqsd0kS3SFkE-W4WWwQbYSwdE&e=>
> (sslServerHandShakeEvent)> (ssl) Go on with the handshake state=5
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLUtils.cc:1510
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslutils.cc-3A1510&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=Qw7t3wWHr5fpoD4lyUqEFXK7FOv3eHdmTYbq_Ppx84g&e=>
> (ssl_callback_info)> (ssl) ssl_callback_info ssl: 0x7ff228028ab0 where:
> 8193 ret: 1 State: SSLv3/TLS write server done
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLUtils.cc:401
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslutils.cc-3A401&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=Vs4SLGP4rxroDg7Rbt7qkTKLaMA6Ya15ER4leCdjttY&e=>
> (ssl_verify_client_callback)> (ssl) Callback: verify client cert
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLNetVConnection.cc:1603
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslnetvconnection.cc-3A1603&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=KJEEYFXd7M33amuR5k3ehukmfIPWBLHNpjXtoK0VPf8&e=>
> (callHooks)> (ssl) callHooks sslHandshakeHookState=5
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLNetVConnection.cc:1675
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslnetvconnection.cc-3A1675&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=7E5pSrM1JfOjHcvAzZcn1oNDErDsn4l9_qZKkXOG3tE&e=>
> (callHooks)> (ssl) callHooks iterated to curHook=(nil)
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLUtils.cc:1510
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslutils.cc-3A1510&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=Qw7t3wWHr5fpoD4lyUqEFXK7FOv3eHdmTYbq_Ppx84g&e=>
> (ssl_callback_info)> (ssl) ssl_callback_info ssl: 0x7ff228028ab0 where:
> 16392 ret: 592 State: error
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLUtils.cc:1510
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslutils.cc-3A1510&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=Qw7t3wWHr5fpoD4lyUqEFXK7FOv3eHdmTYbq_Ppx84g&e=>
> (ssl_callback_info)> (ssl) ssl_callback_info ssl: 0x7ff228028ab0 where:
> 8194 ret: -1 State: error
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLUtils.cc:2344
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslutils.cc-3A2344&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=bYSr4Zeblv-W2acIqCvJDwdjbv4Yd6hfhMfUKGV__Ls&e=>
> (SSLAccept)> (ssl.error.accept) SSL accept returned -1, ssl_error=1,
> ERR_get_error=337100934 (error:1417C086:SSL
> routines:tls_process_client_certificate:certificate verify failed)
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLNetVConnection.cc:1213
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslnetvconnection.cc-3A1213&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=NeldsD4eLgkhoyTV-xMLLQXlHwznW8HxcVoSqwV9Xj0&e=>
> (sslServerHandShakeEvent)> (ssl-diag)
> SSL::140678460933888:error:1417C086:SSL
> routines:tls_process_client_certificate:certificate verify
> failed:../ssl/statem/statem_srvr.c:3701: peer address is 192.168.1.4
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLNetVConnection.cc:1213
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslnetvconnection.cc-3A1213&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=NeldsD4eLgkhoyTV-xMLLQXlHwznW8HxcVoSqwV9Xj0&e=>
> (sslServerHandShakeEvent)> (ssl-diag) SSL handshake error: SSL_ERROR_SSL
> (1), errno=0
> [Jan  6 08:21:22.636] {0x7ff241b10700} DEBUG: <SSLNetVConnection.cc:1339
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__sslnetvconnection.cc-3A1339&d=DwMFAg&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=Ct9BRY8473_NWxRBvSW0ZOHBpyExcu9qgij7MorbZuA&m=eMNTvVzsIr4oMX8X4S6-8igbwFdenlRaU7SzsGbsuN4&s=5R00uQMyJ9YmmMafXRE9xHQzKiK1yEVJf-X57vJ0jGg&e=>
> (sslServerHandShakeEvent)> (ssl-diag)
> SSLNetVConnection::sslServerHandShakeEvent, SSL_ERROR_SSL errno=0
>
>
> The certificate is quite old but valid.
>         Validity
>             Not Before: Aug 18 14:29:55 2017 GMT
>             Not After : Aug 16 14:29:55 2027 GMT
>
> There is nothing noteworthy in diags.log nor any startup errors or
> warnings.
>
> Thanks again,
> Zack
>
>
>
> On Jan 6, 2021, at 7:38 AM, Susan Hinrichs <shinrich@verizonmedia.com>
> wrote:
>
> The config looks good to me.   There are autests that exercise ATS
> requiring certs from the client with a variety of CA verification
> configurations.  I haven't worked with the 8.x branch, but those tests are
> run against that branch I believe.
>
> Is your client certificate directly signed by the certificate in ca.pem?
> Is there an intermediate cert floating around someplace?  Assuming the
> certs are exchanged as in the working case, it seems likely that the
> failure is occurring in the client certificate verification.  I assume that
> ATS is sending the SSL alert.
>
> Any interesting start up failure messages in diags.log?
>
> The next thing to check is turning on debugging and setting the ssl debug
> tag.
>
> On Tue, Jan 5, 2021 at 4:49 PM Zack Bartel <zbartel@proofpoint.com> wrote:
>
>> Hello,
>> We are using client -> TS SSL termination and using client certificates
>> and a CA. While upgrading to 8.x this has stopped working and I'm having
>> trouble figuring out why. I was wondering if there may have been some
>> change in 8.x or error with my config  which would be obvious to someone.
>>
>> The error is "SSL alert number 80" internal error.
>>
>> SSL_connect:SSLv3/TLS read server certificate request
>> SSL_connect:SSLv3/TLS read server done
>> SSL_connect:SSLv3/TLS write client certificate
>> SSL_connect:SSLv3/TLS write client key exchange
>> SSL_connect:SSLv3/TLS write certificate verify
>> SSL_connect:SSLv3/TLS write change cipher spec
>>
>>
>> SSL_connect:SSLv3/TLS write finished
>> read from 0x564393d04170 [0x564393d0e6a3] (5 bytes => 5 (0x5))
>> 0000 - 15 03 03 00 02                                    .....
>> read from 0x564393d04170 [0x564393d0e6a8] (2 bytes => 2 (0x2))
>> 0000 - 02 50                                             .P
>>
>> *SSL3 alert read:fatal:internal error*
>> *SSL_connect:error in error*
>> *139713745085760:error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert
>> internal error:../ssl/record/rec_layer_s3.c:1543:SSL alert number 80*
>>
>>
>>
>> I've inspected the traffic with Wireshark to confirm the exchange is
>> identical between the versions up until the "Internal Error" response.
>> Tried up to 7.1.12-rc0 which works and 8.0.0 -> 8.1.x which does not. I've
>> also tried on Ubuntu and Centos8 with the same results.
>>
>> Relevant config:
>>
>>
>> records.config:
>> CONFIG proxy.config.ssl.CA.cert.path STRING /ssl/
>> CONFIG proxy.config.ssl.CA.cert.filename STRING ca.pem
>>
>> CONFIG proxy.config.ssl.client.verify.server INT 1
>> CONFIG proxy.config.ssl.client.certification_level INT 2
>> CONFIG proxy.config.ssl.client.CA.cert.path STRING /etc/pki/tls/certs/
>> CONFIG proxy.config.ssl.client.CA.cert.filename STRING ca-bundle.crt
>>
>> CONFIG proxy.config.ssl.client.cipher_suite STRING
>> TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DES-CBC3-SHA
>>
>> CONFIG proxy.config.ssl.server.private_key.path STRING /ssl/
>> CONFIG proxy.config.ssl.server.cert.path STRING /ssl/
>>
>> CONFIG proxy.config.ssl.server.cipher_suite STRING
>> ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-DSS-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
>>
>> ssl_multicert.config:
>> ssl_cert_name=server.pem
>>
>>
>> Any help on this greatly appreciated, thank you,
>>
>> Zack
>>
>
>

Mime
View raw message