trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Peach <jpe...@apache.org>
Subject Re: SSL results in segmentation fault
Date Fri, 03 Oct 2014 16:43:38 GMT
On Oct 3, 2014, at 3:32 AM, Matthieu Bienvenüe <matthieu@exultet.net> wrote:

> Any idea to solve this isssu ?

I did a quick test of setting proxy.config.ssl.number.threads to -1, and it didn't crash for
me. Can you enable ssl diagnostics and try again?

CONFIG proxy.config.diags.debug.enabled INT 1
CONFIG proxy.config.diags.debug.tags STRING ssl

> 
> Matthieu
> 
> 
> Le 01/10/2014 09:50, Matthieu Bienvenüe a écrit :
>> 
>> Le 30/09/2014 17:47, Leif Hedstrom a écrit :
>>> 
>>> On Sep 30, 2014, at 9:00 AM, Matthieu Bienvenüe <matthieu@exultet.net>
wrote:
>>> 
>>>> Is that possible to do it on config instead of recompiling ATS ?
>>> 
>>> 
>>> What version are you using? I’m not 100% certain, but I’d expect Geffon’s
additions to not have dedicated SSL threads would avoid the need for that patch as well? Brian?
If I recall, with a recent version of ATS, you’d simply set proxy.config.ssl.number.threads
to -1.
>> When I set this settings SSL don't work and I've the following stack trace : 
>> 
>> NOTE: Traffic Server received Sig 11: Segmentation fault
>> /usr/bin/traffic_server - STACK TRACE: 
>> [0x4001e500]
>> /usr/lib/trafficserver/libtsutil.so.5(ink_hash_table_lookup_entry+0x12)[0x4003c0f2]
>> /usr/lib/trafficserver/libtsutil.so.5(ink_hash_table_lookup+0x24)[0x4003c3b4]
>> /usr/bin/traffic_server[0x8308185]
>> /usr/lib/i386-linux-gnu/i686/cmov/libssl.so.1.0.0(+0x12844)[0x40067844]
>> /usr/lib/i386-linux-gnu/i686/cmov/libssl.so.1.0.0(SSL_accept+0x2a)[0x4008c73a]
>> /usr/bin/traffic_server(_ZN17SSLNetVConnection23sslServerHandShakeEventERi+0x19)[0x8303d89]
>> /usr/bin/traffic_server(_ZN17SSLNetVConnection17sslStartHandShakeEiRi+0x2b)[0x830446b]
>> /usr/bin/traffic_server(_ZN17SSLNetVConnection11net_read_ioEP10NetHandlerP7EThread+0xb30)[0x8305270]
>> /usr/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x27f)[0x830dd7f]
>> /usr/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x98)[0x8339cf8]
>> /usr/bin/traffic_server(_ZN7EThread7executeEv+0x419)[0x833a449]
>> /usr/bin/traffic_server(main+0xf40)[0x80d4e30]
>> /lib/i386-linux-gnu/libc.so.6(__libc_start_main+0xe6)[0x405c9e36]
>> /usr/bin/traffic_server[0x80da229]
>> [TrafficServer] using root directory '/usr'
>> 
>>> 
>>> In either case, why is that patch not committed? Is there a Jira for it?
>>> 
>>> — Leif
>>> 
>>>> 
>>>> Regards,
>>>> 
>>>> Matt
>>>> Le 30/09/2014 16:49, 英才 a écrit :
>>>>> 
>>>>> disable AIO or patch https://github.com/phonehold/with-aio-ssl-init 
may solve your problem
>>>>> 
>>>>> 在 2014年9月30日,下午10:41,Matthieu Bienvenüe <matthieu@exultet.net>
写道:
>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> 
>>>>>> SSL works fine with my certs, but it crashes only after a certain
amount of time/requests.
>>>>>> 
>>>>>> Here is the stack trace from traffic.out:
>>>>>> 
>>>>>> NOTE: Traffic Server received Sig 11: Segmentation fault
>>>>>> /usr/bin/traffic_server - STACK TRACE: 
>>>>>> [0x4001e500]
>>>>>> /usr/bin/traffic_server(_Z12ink_aio_readP11AIOCallbacki+0x2a)[0x830056a]
>>>>>> /usr/bin/traffic_server(_ZN7CacheVC10handleReadEiP5Event+0x282)[0x82c4402]
>>>>>> /usr/bin/traffic_server(_ZN5Cache9open_readEP12ContinuationP7INK_MD5P7HTTPHdrP21CacheLookupHttpConfig13CacheFragTypePci+0x5be)[0x82df68e]
>>>>>> /usr/bin/traffic_server(_ZN14CacheProcessor9open_readEP12ContinuationP3URLbP7HTTPHdrP21CacheLookupHttpConfigl13CacheFragType+0xdc)[0x82c2b4c]
>>>>>> /usr/bin/traffic_server(_ZN11HttpCacheSM18do_cache_open_readEv+0x63)[0x81ab6f3]
>>>>>> /usr/bin/traffic_server(_ZN11HttpCacheSM9open_readEP3URLP7HTTPHdrP21CacheLookupHttpConfigl+0x4c)[0x81aba0c]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM24do_cache_lookup_and_readEv+0x115)[0x81bd105]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x6af)[0x81ce7bf]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x108)[0x81cc8d8]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x300)[0x81c9940]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM23do_api_callout_internalEv+0x54)[0x81c9da4]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x250)[0x81ce360]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x7eb)[0x81ce8fb]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x108)[0x81cc8d8]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x300)[0x81c9940]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM23do_api_callout_internalEv+0x54)[0x81c9da4]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x250)[0x81ce360]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM17handle_api_returnEv+0x108)[0x81cc8d8]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x300)[0x81c9940]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM18state_api_callbackEiPv+0x78)[0x81cc398]
>>>>>> /usr/bin/traffic_server(TSHttpTxnReenable+0x1f0)[0x810ef50]
>>>>>> /usr/lib/trafficserver/modules/stats_over_http.so(+0x102e)[0x4095f02e]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0xd8)[0x81c9718]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM23do_api_callout_internalEv+0x54)[0x81c9da4]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x250)[0x81ce360]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM32state_read_client_request_headerEiPv+0x1e8)[0x81c5738]
>>>>>> /usr/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0x7e)[0x81ca93e]
>>>>>> /usr/bin/traffic_server(_ZN18UnixNetVConnection19readSignalAndUpdateEi+0x45)[0x83166a5]
>>>>>> /usr/bin/traffic_server(_ZN17SSLNetVConnection11net_read_ioEP10NetHandlerP7EThread+0x10b0)[0x83057f0]
>>>>>> /usr/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x27f)[0x830dd7f]
>>>>>> /usr/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x98)[0x8339cf8]
>>>>>> /usr/bin/traffic_server(_ZN7EThread7executeEv+0x419)[0x833a449]
>>>>>> /usr/bin/traffic_server[0x8338ebb]
>>>>>> /lib/i386-linux-gnu/libpthread.so.0(+0x5954)[0x4046b954]
>>>>>> /lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0x40688cbe]
>>>>>> [E. Mgmt] log ==> [TrafficManager] using root directory '/usr'
>>>>>> [TrafficServer] using root directory '/usr'
>>>>>> 
>>>>>> Here is my record.config for SSL parameters:
>>>>>> 
>>>>>> CONFIG proxy.config.http.server_ports STRING 8080 4443:ssl
>>>>>> 
>>>>>> CONFIG proxy.config.ssl.enabled INT 1
>>>>>> CONFIG proxy.config.ssl.server.cert.path STRING /etc/trafficserver/ssl/
>>>>>> CONFIG proxy.config.ssl.server.private_key.path STRING /etc/trafficserver/ssl/
>>>>>> 
>>>>>> And for ssl_multicert.config: 
>>>>>> 
>>>>>> ssl_cert_name=new2014/100.pem ssl_key_name=new2014/100.key
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Le 30/09/2014 15:54, Susan Hinrichs a écrit :
>>>>>>> Matt, 
>>>>>>> 
>>>>>>> Is there a basic stack trace in traffic.out?   What is your SSL
configuration?  Do you have certs set up in ssl_multicert.config? Or are you doing a blind
tunnel on the SSL traffic? 
>>>>>>> 
>>>>>>> Susan 
>>>>>>> 
>>>>>>> On 9/30/2014 2:14 AM, Matthieu Bienvenüe wrote: 
>>>>>>>> Hello ! 
>>>>>>>> 
>>>>>>>> I'm configuring ATS as a reverse proxy and I need SSL support.

>>>>>>>> 
>>>>>>>> ATS runs on OpenVZ on Debian. It's the version 5.0.1 installed
from backport packages. 
>>>>>>>> 
>>>>>>>> ATS works fine, SSL too. But after a while SSL makes ATS
crash. 
>>>>>>>> 
>>>>>>>> In manager.log I found that there is a segmentation fault:

>>>>>>>> 
>>>>>>>> [Sep 29 16:08:33.020] Manager {0xb6fb76d0} ERROR: [LocalManager::pollMgmtProcessServer]
Server Process terminated due to Sig 11: Segmentation fault 
>>>>>>>> [Sep 29 16:08:33.021] Manager {0xb6fb76d0} ERROR: [Alarms::signalAlarm]
Server Process was reset 
>>>>>>>> [Sep 29 16:08:34.041] Manager {0xb6fb76d0} NOTE: [LocalManager::startProxy]
Launching ts process 
>>>>>>>> [Sep 29 16:08:34.049] Manager {0xb6fb76d0} NOTE: [LocalManager::pollMgmtProcessServer]
New process connecting fd '16' 
>>>>>>>> [Sep 29 16:08:34.049] Manager {0xb6fb76d0} NOTE: [Alarms::signalAlarm]
Server Process born 
>>>>>>>> 
>>>>>>>> Here is a dump of the syslog when crashing: 
>>>>>>>> 
>>>>>>>> Sep 30 07:05:09 ats traffic_manager[5471]: {0xb704d6d0} FATAL:
[LocalManager::pollMgmtProcessServer] Error in read (errno: 104) 
>>>>>>>> Sep 30 07:05:09 ats traffic_manager[5471]: {0xb704d6d0} ERROR:
[LocalManager::sendMgmtMsgToProcesses] Error writing message 
>>>>>>>> Sep 30 07:05:09 ats traffic_manager[5471]: {0xb704d6d0} ERROR:
(last system error 32: Broken pipe) 
>>>>>>>> Sep 30 07:05:09 ats traffic_cop[23694]: cop received child
status signal [5471 256] 
>>>>>>>> Sep 30 07:05:09 ats traffic_cop[23694]: traffic_manager not
running, making sure traffic_server is dead 
>>>>>>>> Sep 30 07:05:09 ats traffic_cop[23694]: spawning traffic_manager

>>>>>>>> Sep 30 07:05:09 ats traffic_manager[6938]: NOTE: --- Manager
Starting --- 
>>>>>>>> Sep 30 07:05:09 ats traffic_manager[6938]: NOTE: Manager
Version: Apache Traffic Server - traffic_manager - 5.0.1 - (build # 7259 on Aug 25 2014 at
09:26:11) 
>>>>>>>> Sep 30 07:05:09 ats traffic_manager[6938]: NOTE: Unable to
set RLIMIT_NOFILE(7):cur(1475961),max(1475961) 
>>>>>>>> Sep 30 07:05:09 ats traffic_manager[6938]: NOTE: RLIMIT_NOFILE(7):cur(30000),max(30000)

>>>>>>>> Sep 30 07:05:09 ats traffic_manager[6938]: ERROR ==> [runAsUser]
Error: Failed to restore capabilities after switch to user trafficserver. 
>>>>>>>> Sep 30 07:05:11 ats traffic_server[6946]: NOTE: --- traffic_server
Starting --- 
>>>>>>>> Sep 30 07:05:11 ats traffic_server[6946]: NOTE: traffic_server
Version: Apache Traffic Server - traffic_server - 5.0.1 - (build # 7259 on Aug 25 2014 at
09:27:18) 
>>>>>>>> Sep 30 07:05:11 ats traffic_server[6946]: NOTE: Unable to
set RLIMIT_NOFILE(7):cur(-611778560),max(-611778560) 
>>>>>>>> Sep 30 07:05:13 ats traffic_manager[6938]: {0xb708b6d0} ERROR:
[LocalManager::pollMgmtProcessServer] Server Process terminated due to Sig 11: Segmentation
fault 
>>>>>>>> Sep 30 07:05:13 ats traffic_manager[6938]: {0xb708b6d0} ERROR:
[Alarms::signalAlarm] Server Process was reset 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Any idea where to look for to solve this problem ? 
>>>>>>>> 
>>>>>>>> Thanks a lot ! 
>>>>>>>> 
>>>>>>>> Matt 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 


Mime
View raw message