trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lei Sun <lei....@gmail.com>
Subject Re: [E] https issue
Date Sat, 05 Dec 2020 06:18:59 GMT
Hi Guys,

I followed the instructions from Alan, and set up the certs.

> dest_ip=*   ssl_cert_name=host.pem ssl_key_name=host.key


then I followed the instructions from Susan and tested with --resolve

curl -k -v --resolve 'httbin.org:8443:127.0.0.1'
> https://httpbin.org/get?answer=42 -v


It seems that the ats still lets curl create tls handshake directly with
the origin server, which isn't what I'm looking for.

> $ curl -k -v --resolve 'httbin.org:8443:127.0.0.1'
> https://httpbin.org/get?answer=42 -v
> * Added httbin.org:8443:127.0.0.1 to DNS cache
> *   Trying 184.72.216.47...
> * TCP_NODELAY set
> * Connected to httpbin.org (184.72.216.47) port 443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> *   CAfile: /etc/ssl/cert.pem
>   CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):


I'm trying to do the following, notice the "origin": "*103.253.27.108*",
except that I want to make the request to ATS on *127.0.0.1*, then have the
ATS forward this request to *103.253.27.108*, and return the *https*
response from *103.253.27.108* back to the client. How can we make this
happen with ATS please?

$ curl --proxy http://103.253.27.108:80 *https*://httpbin.org/get?answer=42
> -v
> *   Trying 103.253.27.108...
> * TCP_NODELAY set
> * Connected to 103.253.27.108 (103.253.27.108) port 80 (#0)
> * Establish HTTP proxy tunnel to httpbin.org:443
> > CONNECT httpbin.org:443 HTTP/1.1
> > Host: httpbin.org:443
> > User-Agent: curl/7.54.0
> > Proxy-Connection: Keep-Alive
> >
> < HTTP/1.1 200 Connection established
> <
> * Proxy replied OK to CONNECT request
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> *   CAfile: /etc/ssl/cert.pem
>   CApath: none
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Client hello (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS change cipher, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
> * ALPN, server accepted to use h2
> * Server certificate:
> *  subject: CN=httpbin.org
> *  start date: Jan 18 00:00:00 2020 GMT
> *  expire date: Feb 18 12:00:00 2021 GMT
> *  subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
> *  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
> *  SSL certificate verify ok.
> * Using HTTP2, server supports multi-use
> * Connection state changed (HTTP/2 confirmed)
> * Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
> len=0
> * Using Stream ID: 1 (easy handle 0x7fa826808c00)
> > GET /get?answer=42 HTTP/2
> > Host: httpbin.org
> > User-Agent: curl/7.54.0
> > Accept: */*
> >
> * Connection state changed (MAX_CONCURRENT_STREAMS updated)!
> < HTTP/2 200
> < date: Sat, 05 Dec 2020 06:05:52 GMT
> < content-type: application/json
> < content-length: 288
> < server: gunicorn/19.9.0
> < access-control-allow-origin: *
> < access-control-allow-credentials: true
> <
> {
>   "args": {
>     "answer": "42"
>   },
>   "headers": {
>     "Accept": "*/*",
>     "Host": "httpbin.org",
>     "User-Agent": "curl/7.54.0",
>     "X-Amzn-Trace-Id": "Root=1-5fcb2340-1bc9172e5a20c5b55ee319c6"
>   },
>   "origin": "*103.253.27.108*",
>   "url": "https://httpbin.org/get?answer=42"
> }
> * Connection #0 to host 103.253.27.108 left intact




On Thu, Dec 3, 2020 at 9:34 PM Lei Sun <lei.sun@gmail.com> wrote:

> Guess what, I just realized that all three of us are Yahoo Alums LOL
>
> On Thu, Dec 3, 2020 at 9:31 PM Lei Sun <lei.sun@gmail.com> wrote:
>
>> This is super helpful! Thank you so much Alan and Susan!
>>
>> On Thu, Dec 3, 2020 at 1:36 PM Susan Hinrichs <shinrich@verizonmedia.com>
>> wrote:
>>
>>> The --resolve option is very helpful for using curl to direct requests
>>> to the proxy to terminate.
>>>
>>> curl -k -v --resolve 'httbin.org:4443:127.0.0.1'
>>> https://httpbin.org:4443/get?answer=4a
>>>
>>> Adding the -k assuming you are using a self-signed cert in ATS for
>>> testing.  Also assuming your ATS is configured to listen for TLS on port
>>> 4443 in this example.
>>>
>>> On Thu, Dec 3, 2020 at 1:29 PM Alan Carroll <
>>> solidwallofcode@verizonmedia.com> wrote:
>>>
>>>> You will need to set up the certificates for ATS in that case. Although
>>>> it is possible to do this in "records.config", that is (IMHO) deprecated
>>>> because it has been superceded by "ssl_multicert.config". I would start
>>>> with that directly, it will be easier.
>>>>
>>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/files/ssl_multicert.config.en.html
>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.trafficserver.apache.org_en_latest_admin-2Dguide_files_ssl-5Fmulticert.config.en.html&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=GAqM_xZpxNbVqsR-aGvQBjOG3d33Y2-i4ynL-JkEouY&m=7bubaML3NbE3IHatnbQQrdelwW56A0ptWueuP2dNGGU&s=L4iVe956pejQQnlOZXgP2jUNJ85P-HmFam8gu5eji0U&e=>
>>>> If there is only a single certificate, you will want to use "dest_ip=*"
>>>> as the configuration which will use that certificate for all outbound
>>>> connections.
>>>> You'll need to use different options to curl to test this, as using
>>>> "--proxy" with an "https" destination will always bypass TLS on the proxy.
>>>>
>>>>
>>>> On Thu, Dec 3, 2020 at 1:03 PM Lei Sun <lei.sun@gmail.com> wrote:
>>>>
>>>>> Hi Alan,
>>>>>
>>>>> Thanks for responding! Yes, I learned about the CONNECT method.
>>>>>
>>>>> I can confirm that for the "https" method, I don't want the client to
>>>>> do TLS directly with the server. Instead, I'd like the trafficserver
to
>>>>> take that request, decrypt it, then pretend to be a client, and do TLS
on
>>>>> behalf of the client with the upstream.
>>>>>
>>>>> Thanks,
>>>>> Lei
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 3, 2020 at 12:13 PM Alan Carroll <
>>>>> solidwallofcode@verizonmedia.com> wrote:
>>>>>
>>>>>> The first thing to note is "curl --proxy" behaves very differently
if
>>>>>> the target URL is "http" vs. "https". In the former case, curl will
do a
>>>>>> TCP network connection to the proxy and then sends a GET request.
In the
>>>>>> latter case, as you can see from the output, it does a TCP network
>>>>>> connection to the proxy and then sends a CONNECT (not a GET) to the
proxy.
>>>>>> After this, curl will do TLS negotiation with the upstream, NOT with
ATS.
>>>>>> It is unclear from your description if this is what you want.
>>>>>>
>>>>>> So, first question - should ATS do TLS negotiation with the user
>>>>>> agent and decrypt the request? Or should it just do a blind tunnel
pass the
>>>>>> raw bytes to the upstream so the upstream does the TLS with the user
agent?
>>>>>>
>>>>>> On Wed, Dec 2, 2020 at 5:41 PM Lei Sun <lei.sun@gmail.com>
wrote:
>>>>>>
>>>>>>> Hi Kit,
>>>>>>>
>>>>>>> I'm trying to set up the trafficserver as an intermediary forward
>>>>>>> proxy.
>>>>>>>
>>>>>>> For example,
>>>>>>> 1) http client send request to trafficserver.
>>>>>>> 2) trafficserver then pass this request to the downstream proxy
>>>>>>> 3) downstream proxy then route this request to the origin site
>>>>>>> 4) origin site send data back to the downstream proxy
>>>>>>> 5) downstream proxy send data back to trafficserver
>>>>>>> 6) trafficserver send data back to the http client.
>>>>>>>
>>>>>>> I was able to make the entire request chain work if the origin
site
>>>>>>> serves content directly through HTTP.
>>>>>>>
>>>>>>>> curl --proxy *http*://127.0.0.1:8080
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>>>> *http*://httpbin.org/get?answer=4a
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>>>> -v
>>>>>>>
>>>>>>>
>>>>>>> However, I ran into issues when I was trying to request for content
>>>>>>> served from HTTPS.
>>>>>>>
>>>>>>> $ curl --proxy *http*://127.0.0.1:8080
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8080&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=so4reBHk8fjNcUgFl5Rl6jW1O795FlyMKH-HBzls_yE&e=>
>>>>>>>> *https*://httpbin.org/get?answer=4a
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org_get-3Fanswer-3D4a&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=Ytt8BUmEjADeQ8BgYA33Srb-fKblsmhS0lnnYYA-7H4&e=>
>>>>>>>> -v
>>>>>>>> *   Trying 127.0.0.1...
>>>>>>>> * TCP_NODELAY set
>>>>>>>> * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
>>>>>>>> * Establish HTTP proxy tunnel to httpbin.org:443
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>>> > CONNECT httpbin.org:443
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>>> HTTP/1.1
>>>>>>>> > Host: httpbin.org:443
>>>>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__httpbin.org-3A443&d=DwMFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=j_aUSUG3Sl8woHSXT1UiCQWjpkvw7qS5UAl79s5x2TQ&s=QVzyvEwxJgKDXVbGsQTcSau-LJcP5X22mrrpyKfksAY&e=>
>>>>>>>> > User-Agent: curl/7.54.0
>>>>>>>> > Proxy-Connection: Keep-Alive
>>>>>>>> >
>>>>>>>> < HTTP/1.1 200 OK
>>>>>>>> < Date: Wed, 02 Dec 2020 20:53:31 GMT
>>>>>>>> < Proxy-Connection: keep-alive
>>>>>>>> < Server: ATS/10.0.0
>>>>>>>> <
>>>>>>>> * Proxy replied OK to CONNECT request
>>>>>>>> * ALPN, offering h2
>>>>>>>> * ALPN, offering http/1.1
>>>>>>>> * Cipher selection:
>>>>>>>> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>>>>>>>> * successfully set certificate verify locations:
>>>>>>>> *   CAfile: /etc/ssl/cert.pem
>>>>>>>>   CApath: none
>>>>>>>> * *TLSv1.2 (OUT), TLS handshake, Client hello (1):*
>>>>>>>> * error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
version
>>>>>>>> number
>>>>>>>> * stopped the pause stream!
>>>>>>>> * Closing connection 0
>>>>>>>> curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong
>>>>>>>> version number
>>>>>>>>
>>>>>>>
>>>>>>> From the error message, it seems that curl was able to connect
to
>>>>>>> the origin server, and even attempted to send the initial TLS
handshake,
>>>>>>> but somehow the handshake wasn't completed.
>>>>>>>
>>>>>>> Here are my questions.
>>>>>>> 1) What's likely the cause?
>>>>>>> 2) How can I fix it.
>>>>>>>
>>>>>>> Thank you!
>>>>>>> Lei
>>>>>>>
>>>>>>>

-- 
Stay Hungry, Stay Foolish.
Lei Sun
Cell: 408-306-9199

Mime
View raw message