trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jack Bates <duh...@nottheoilrig.com>
Subject Re: 回复:Re: Openssl 1.1.0f Support
Date Tue, 26 Sep 2017 22:44:18 GMT
Cool -- I replied to you pull request. Thanks!

On 25/09/17 04:56 PM, Chou, Peter wrote:
> Jack,
>
> Regarding compiling 6.2.x with OpenSSL 1.1.0f, I submitted PR #2570 which back-ports
your fix for the sslheaders plugin that was required due to X509 being made opaque in OpenSSL
1.1. There was also a linking issue with Ubuntu when compiling with the 'with-openssl' configure
option. Apparently, this might not occur on Red Hat which is what Jeremy used below.
>
> Thanks,
> Peter
>
> -----Original Message-----
> From: Jeremy Payne [mailto:jp557198@gmail.com]
> Sent: Friday, September 22, 2017 10:52 AM
> To: users@trafficserver.apache.org
> Subject: Re: 回复:Re: Openssl 1.1.0f Support
>
> openssl lib and include directories werent matching versions.
> fixed that, and was able to compile without issue.
>
>
>
>
>
> On Fri, Sep 22, 2017 at 11:28 AM, Jack Bates <duhapa@nottheoilrig.com> wrote:
>> I remember backporting some fixes for building with OpenSSL 1.1 to 6.2 [1]
>> ... Which errors are you still getting?
>>
>> I tried just now and successfully built the 6.2 branch with OpenSSL 1.1.0f
>> -- I did get warnings but no errors:
>>
>>   $ git clone -b 6.2.x https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver.git&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=8HrQiBnIzItF-b6piI1aWqjSdcpsW9r59uuAoKA_sIQ&e=
>>   $ cd trafficserver
>>   $ autoreconf -i
>>   $ ./configure
>>   $ make
>>
>> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_trafficserver_pull_1321&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=nr0xsT0XCw7uLhrHvYwBhmQd5BbnSAKC2RfefCTpHgI&e=
>>
>>
>> On 22/09/17 06:44 AM, Jeremy Payne wrote:
>>>
>>> Did you have to make any changes to 6.2 for it to compile cleanly
>>> against openssl 1.1 ?
>>> I can only get ATS 7+ to compile without producing any errors.
>>>
>>>
>>>
>>> On Thu, Sep 21, 2017 at 10:05 PM,  <iloveperl@sina.cn> wrote:
>>>>
>>>> I tested traffic server 6.2 + openssl 1.1 and traffic server 6.2 +
>>>> openssl
>>>> 1.0.1 + patch respectively, and they almost have the same performance.
>>>>
>>>>
>>>> ----- 原始邮件 -----
>>>> 发件人:<iloveperl@sina.cn>
>>>> 收件人:"bcall" <bcall@apache.org>
>>>> 抄送人:"users" <users@trafficserver.apache.org>
>>>> 主题:回复:Re: Openssl 1.1.0f Support
>>>> 日期:2017年09月22日 10点55分
>>>>
>>>> With the patch, the ERR_clear_error() will only be called when the error
>>>> occurs. In the normal situation, ERR_clear_error() will not be called, so
>>>> the Err_get_state() will not be called and no lock contention in openssl
>>>> 1.0.1 with the patch.
>>>>
>>>>
>>>> ----- 原始邮件 -----
>>>> 发件人:Bryan Call <bcall@apache.org>
>>>> 收件人:iloveperl@sina.cn
>>>> 抄送人:users <users@trafficserver.apache.org>
>>>> 主题:Re: Openssl 1.1.0f Support
>>>> 日期:2017年09月21日 23点37分
>>>>
>>>> This only changes the order of the calls.  There is still going to be
>>>> lock
>>>> contention inside OpenSSL 1.0.1.
>>>>
>>>> -Bryan
>>>>
>>>> On Sep 20, 2017, at 11:37 PM, iloveperl@sina.cn wrote:
>>>>
>>>>
>>>> The following traffic server patch can improve openssl 1.0.1 performance
>>>> as
>>>> openssl 1.1.0:
>>>>
>>>> diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
>>>> index 5c9709c..5d306a1 100644
>>>> --- a/iocore/net/SSLUtils.cc
>>>> +++ b/iocore/net/SSLUtils.cc
>>>> @@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
>>>> nbytes, int64_t &nwritten)
>>>>    if (unlikely(nbytes == 0)) {
>>>>      return SSL_ERROR_NONE;
>>>>    }
>>>> -  ERR_clear_error();
>>>> +
>>>>    int ret = SSL_write(ssl, buf, (int)nbytes);
>>>>    if (ret > 0) {
>>>>      nwritten = ret;
>>>> @@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t
>>>> nbytes, int64_t &nwritten)
>>>>      ERR_error_string_n(e, buf, sizeof(buf));
>>>>      Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d,
>>>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
>>>>    }
>>>> +
>>>> +  ERR_clear_error();
>>>> +
>>>>    return ssl_error;
>>>>  }
>>>>
>>>> @@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes,
>>>> int64_t &nread)
>>>>    if (unlikely(nbytes == 0)) {
>>>>      return SSL_ERROR_NONE;
>>>>    }
>>>> -  ERR_clear_error();
>>>> +
>>>>    int ret = SSL_read(ssl, buf, (int)nbytes);
>>>>    if (ret > 0) {
>>>>      nread = ret;
>>>> @@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t
>>>> nbytes,
>>>> int64_t &nread)
>>>>      Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d,
>>>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
>>>>    }
>>>>
>>>> +  ERR_clear_error();
>>>> +
>>>>    return ssl_error;
>>>>  }
>>>>
>>>>  ssl_error_t
>>>>  SSLAccept(SSL *ssl)
>>>>  {
>>>> -  ERR_clear_error();
>>>>    int ret = SSL_accept(ssl);
>>>>    if (ret > 0) {
>>>>      return SSL_ERROR_NONE;
>>>> @@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
>>>>      Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d,
>>>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
>>>>    }
>>>>
>>>> +  ERR_clear_error();
>>>> +
>>>>    return ssl_error;
>>>>  }
>>>>
>>>>  ssl_error_t
>>>>  SSLConnect(SSL *ssl)
>>>>  {
>>>> -  ERR_clear_error();
>>>>    int ret = SSL_connect(ssl);
>>>>    if (ret > 0) {
>>>>      return SSL_ERROR_NONE;
>>>> @@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
>>>>      Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d,
>>>> ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
>>>>    }
>>>>
>>>> +  ERR_clear_error();
>>>> +
>>>>    return ssl_error;
>>>>  }
>>>>
>>>>
>>>> From: Bryan Call <bcall@apache.org>
>>>> Reply-To: "users@trafficserver.apache.org"
>>>> <users@trafficserver.apache.org>
>>>> Date: Thursday, September 21, 2017 at 8:38 AM
>>>> To: "users@trafficserver.apache.org" <users@trafficserver.apache.org>
>>>> Subject: Re: Openssl 1.1.0f Support
>>>>
>>>>
>>>>
>>>> I meant to say 1.1.0.
>>>>
>>>>
>>>>
>>>> -Bryan
>>>>
>>>>
>>>>
>>>> On Sep 20, 2017, at 3:54 PM, Bryan Call <bcall@apache.org> wrote:
>>>>
>>>>
>>>>
>>>> I was see something like 2x the performance in my benchmarks with OpenSSL
>>>> 1.0.1.  I have been doing all my development with OpenSSL 1.0.1 ATS since
>>>> May, when I upgraded to Fedora 26.
>>>>
>>>>
>>>>
>>>> -Bryan
>>>>
>>>>
>>>>
>>>> On Sep 20, 2017, at 2:16 PM, Dave Thompson <davet@oath.com> wrote:
>>>>
>>>>
>>>>
>>>> Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by
>>>> now
>>>> at best.   The gist of my recollection is that QAT is an IO based async
>>>> engine, which of course ATS already has done extensively.   I recall the
>>>> under-the-hood QAT longjumping was a non-starter in an ATS framework.
>>>> This
>>>> was all static code analysis.  Integration looked like a non-starter, so
>>>> no
>>>> performance test done.
>>>>
>>>> Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard
>>>> aes-ni
>>>> acceleration", Susan (?Bryan?) was just telling me today of a measured
>>>> order
>>>> of magnitude improvement over with the same using Openssl 1.0.1(x) and
>>>> small
>>>> packet sizes...  Improvement attributed to lock contention issues in the
>>>> older OpenSSL 1.0.1(x).
>>>>
>>>>
>>>>
>>>> Dave
>>>>
>>>>
>>>>
>>>> On Wed, Sep 20, 2017 at 3:22 PM, Jeremy Payne <jp557198@gmail.com>
wrote:
>>>>
>>>> Dave,
>>>>
>>>>
>>>> Did you run any comparison performance tests using the QAT engine ?
>>>>
>>>> Specifically around these configurations(or similar)
>>>>
>>>>
>>>> 1. ATS + Openssl 1.1.0(x) + QAT engine(sync)
>>>>
>>>> 2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Sep 20, 2017 at 11:26 AM, Dave Thompson <davet@oath.com> wrote:
>>>>
>>>>> July 2016, I was evaluating the async Quick Assist in the context of
>>>>> ATS,
>>>>
>>>>
>>>>> and came away with the opinion it's value comes into play with a much
>>>>
>>>>
>>>>> simpler application.   It's effectively it's own async engine, long
>>>>> jumping
>>>>
>>>>
>>>>> across the stack, and doesn't play well or add  value to ATS's more
>>>>
>>>>
>>>>> extensive model to do similar.... not to mention mutually exclusive in
>>>>> their
>>>>
>>>>
>>>>> current forms.
>>>>
>>>>
>>>>>
>>>>
>>>>> Dave
>>>>
>>>>
>>>>>
>>>>
>>>>>
>>>>
>>>>>
>>>>
>>>>> On Wed, Sep 20, 2017 at 10:08 AM, Alan Carroll
>>>>> <solidwallofcode@oath.com>
>>>>
>>>>
>>>>> wrote:
>>>>
>>>>
>>>>>>
>>>>
>>>>>> Susan and Dave Thompson were working on something related to that,
>>>>>> "crypto
>>>>
>>>>
>>>>>> proxy". There's a small mention of it by Susan at the Fall 2016 summit
>>>>>> in
>>>>
>>>>
>>>>>> the TLS state slides
>>>>
>>>>
>>>>>> (https://urldefense.proofpoint.com/v2/url?u=https-3A__cwiki.apache.org_confluence_display_TS_Presentations-2B-2D-2B2016&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=E1it3OczMHrBWqN3XLenMT3yqV9H6AxFkXEuxM991cw&s=zhE5_TgdGKvMRHbD7pN1VeJPtK8Zzlu8mGsIu7xpNVM&e=
).
>>>>>> I'd
>>>>
>>>>
>>>>>> start there and see if you can bug Susan or Good Dave*. Although
that
>>>>>> work
>>>>
>>>>
>>>>>> was designed to use an off box crypto engine, the implementation
from
>>>>>> the
>>>>
>>>>
>>>>>> ATS point of view is identical to what you're writing about. Susan
will
>>>>>> be
>>>>
>>>>
>>>>>> at the Fall 2017 Summit, I'd look her up then as well.
>>>>
>>>>
>>>>>>
>>>>
>>>>>> * To distinguish from "Evil Dave" Carlin.
>>>>
>>>>
>>>>>>
>>>>
>>>>>> On Wed, Sep 20, 2017 at 9:44 AM, Jeremy Payne <jp557198@gmail.com>
>>>>>> wrote:
>>>>
>>>>
>>>>>>>
>>>>
>>>>>>> Thanks guys.. Thats all I needed to know.. Now I can look closer
at my
>>>>
>>>>
>>>>>>> end. Will let you know what I find.
>>>>
>>>>
>>>>>>>
>>>>
>>>>>>> Also, any plans on supporting openssl async, which then allows
for
>>>>
>>>>
>>>>>>> taking full advantage of the Intel QAT engine?
>>>>
>>>>
>>>>>>> Understood patches/commits are welcome, but just figured there
may be
>>>>
>>>>
>>>>>>> some behind the scene works already started.
>>>>
>>>>
>>>>>>>
>>>>
>>>>>>> Thanks!
>>>>
>>>>
>>>>>>>
>>>>
>>>>>>> On Tue, Sep 19, 2017 at 6:31 PM, Alan Carroll
>>>>>>> <solidwallofcode@oath.com>
>>>>
>>>>
>>>>>>> wrote:
>>>>
>>>>
>>>>>>>> Susan has also run some performance tests with 7.1.x and
openSSL 1.1
>>>>
>>>>
>>>>>>>> vs.
>>>>
>>>>
>>>>>>>> openSSL 1.0.2.
>>>>
>>>>
>>>>>>>>
>>>>
>>>>>>>> On Tue, Sep 19, 2017 at 5:55 PM, Leif Hedstrom <zwoop@apache.org>
>>>>
>>>>
>>>>>>>> wrote:
>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>> On Sep 19, 2017, at 2:20 PM, Jeremy Payne <jp557198@gmail.com>
>>>>>>>>> wrote:
>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>> I can link ATS 7.x and 8.x against openssl 1.1.0f, however,
for some
>>>>
>>>>
>>>>>>>>> reason I can't establish a SSL/TLS connection.  Has anyone
>>>>
>>>>
>>>>>>>>> successfully linked ATS against openssl 1.1.0f  and successfully
>>>>>>>>> been
>>>>
>>>>
>>>>>>>>> able to establish a SSL/TLS session?
>>>>
>>>>
>>>>>>>>> In other words, is openssl 1.1.0f supported by ATS? If
not, what
>>>>>>>>> about
>>>>
>>>>
>>>>>>>>> an earlier version of 1.1.0(x)??
>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>> Yeh, we’re running current master with OpenSSL v1.1.0f
on
>>>>>>>>> docs.trafficserver.apache.org. Maybe you have some mismatch
/ issues
>>>>
>>>>
>>>>>>>>> between
>>>>
>>>>
>>>>>>>>> headers (when compiling ATS) and runtime?
>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>> Cheers,
>>>>
>>>>
>>>>>>>>>
>>>>
>>>>>>>>> — Leif
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>

Mime
View raw message