trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Susan Hinrichs <shinr...@oath.com>
Subject Re: Traffic server suddenly drop the packets and crashed
Date Fri, 16 Nov 2018 17:16:28 GMT
You probably want to extend your ip_local_port range to 1025 to 65535.  I
assume you have an application that requires the reserved ports.

When you run your test, what is the output of "ss -s" ?  Is the number of
ports in TIME_WAIT really high?



On Fri, Nov 16, 2018 at 8:43 AM Vasanth Mathivanan <
vasanth.m@evolutiondigital.com> wrote:

> Yes , its generating core dump  file in traffic server root path and we
> are using ATS.6.2 in centos 6.5.
>
> traffic.out logs
>
> *“ATAL: InkAPI.cc:1879: failed assert `sdk_sanity_check_mbuffer(bufp) ==
> TS_SUCCESS`*
>
> *traffic_server: using root directory '/opt/trafficserver'*
>
> *traffic_server: received signal 6 (Aborted)*
>
> *traffic_server - STACK TRACE: *
>
>
> */opt/trafficserver/bin/traffic_server(_Z19crash_logger_invokeiP7siginfoPv+0x8e)[0x4ac5ee]*
>
> */lib64/libpthread.so.0[0x3030e0f7e0]*
>
> */lib64/libc.so.6(gsignal+0x35)[0x3030a32495]*
>
> */lib64/libc.so.6(abort+0x175)[0x3030a33c75]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x28228)[0x2b5cb60dc228]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x282bc)[0x2b5cb60dc2bc]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x26595)[0x2b5cb60da595]*
>
> */opt/trafficserver/bin/traffic_server(TSHandleMLocRelease+0x3a)[0x4c33ca]*
>
>
> */opt/trafficserver/libexec/trafficserver/stale_while_revalidate.so(+0x38c9)[0x2b5e9eaa48c9]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN15INKContInternal12handle_eventEiPv+0xdb)[0x4c12ab]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x1b7)[0x5ac967]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x65b)[0x5b226b]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM22state_cache_open_writeEiPv+0x20e)[0x5a3cde]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xca)[0x5b3f3a]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN11HttpCacheSM22state_cache_open_writeEiPv+0x281)[0x58a481]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x766f00]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN7EThread7executeEv+0x811)[0x767bd1]*
>
> */opt/trafficserver/bin/traffic_server[0x76698a]*
>
> */lib64/libpthread.so.0[0x3030e07aa1]*
>
> */lib64/libc.so.6(clone+0x6d)[0x3030ae8bdd]”*
>
>
>
> No I did not set ephemeral ports .
>
>
> Had default values already .
>
> net.ipv4.ip_local_port_range = 32768     60999
>
> net.ipv4.ip_local_reserved_ports =                    ###this one is empty
>
>
>
> So its set by manually .
>
> net.ipv4.ip_local_reserved_ports = 30000-31000
>
>
>
> Is this ok ? .
>
>
>
>
> __Vasanth
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Susan Hinrichs <shinrich@oath.com>
> *Sent: *Friday, November 16, 2018 7:55 PM
> *To: *users@trafficserver.apache.org
> *Subject: *Re: Traffic server suddenly drop the packets and crashed
>
>
> Are you exhausting ephemeral ports?  What is the output of  "ss -s" on
> your Traffic Server machine? That is quite easy exhaust all of the
> ephemeral ports for a single load testing client machine between active and
> time-wait connections.
>
> Is Traffic Server generating a core or a stack trace?  What version of
> Traffic Server?
>
> On Fri, Nov 16, 2018 at 1:42 AM Vasanth Mathivanan <
> vasanth.m@evolutiondigital.com> wrote:
>
>> Hi ,
>>
>> We are testing 1000 client access one by one load test after  700 clients
>> dropping the packets like “connection timeout” or “Connection reset by
>> peer ” errors .
>>
>>
>>
>> Ram and Cpus resources are everything is fine .Ethernet speed also 20gb/s
>> both client  and traffic server  . Origin server has 2gb/s speed between
>> ATS and Origin server .While running 700 requests in origin server content
>> will be  receiving speed less than 10Mb/s then traffic server and client
>> getting more than 1Gb/s speed .
>>
>>
>>
>> We are also monitoring that resources everything running fine but crashed
>> the traffic server after 700 requests  .
>>
>>
>>
>> Note: We are created Java player , We will give the no of  requests its
>> call the particular URL based on given requests .
>>
>>
>>
>> __Vasanth
>>
>>
>>
>> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>>
>>
>

Mime
View raw message