trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Payne <jp557...@gmail.com>
Subject Re: Ramdisk - 6.2.1 vs 7.0.0
Date Sat, 10 Jun 2017 15:37:33 GMT
ATS is configured to listen on 443:ssl. But there is no certificate called
in ssl_multicert or other. So https requests
will fail. Plus we are not sending https requests during these test runs.


On Sat, Jun 10, 2017 at 10:11 AM, Bryan Call <bcall@apache.org> wrote:

> Do you have TLS connections? (https).  You can use tcmalloc or jemalloc to
> profile the memory allocation.
>
> -Bryan
>
> On Jun 9, 2017, at 11:50 AM, Chou, Peter <pbchou@labs.att.com> wrote:
>
> Bryan,
>
> Our default_inactivity_timeout is set to 300s. We have looked at the
> memory dump in traffic.out as well, but we’re stumped because the memory
> that is tracked in the dump only comes out to about 8GB and 6GB total
> allocated and used respectively while pmap -x <pid> reports around 26GB
> memory allocated to the process.
>
> From pmap -- 26127468 11646528 11646094    8773892  11644512 12341772 ==
> [ size rss pss referenced anonymous swap ] (KB)
> From traffic.out -- 7895071920 |         5956998576 |            | TOTAL
> (B)
>
> Is this similar to run-away anon region allocation seen before? Appreciate
> any insights.
>
> Thanks,
> Peter
>
>
> *From:* Bryan Call [mailto:bcall@apache.org <bcall@apache.org>]
> *Sent:* Friday, June 09, 2017 9:02 AM
> *To:* users@trafficserver.apache.org
> *Subject:* Re: Ramdisk - 6.2.1 vs 7.0.0
>
> What is your proxy.config.net.default_inactivity_timeout set to?  You can
> also do a dump of the memory by doing:
>
> $ sudo kill -SIGUSR1 $(pidof traffic_server). # and look at traffic.out.
>
> That will give a pretty good idea on where the memory is being used.
>
> -Bryan
>
>
> On Jun 8, 2017, at 3:35 PM, Jeremy Payne <jp557198@gmail.com> wrote:
>
> ATS versions - 6.2.1
> Cache disk - 18G ramdisk - /dev/ram0
> Storage.config configured to use 14G of ramdisk
> System memory - 32G
> Traffic type - HLS Live
> Connections - sustained 4k
> Trafffic - 5-6gbps
> Hit ratio - 97%
>
> So it looks like I misread/misinterpreted a few things during testing
> referenced in my previous email.
> It looks like my issue was never run away memory using ramdisk,
> it looks like something internal to ATS that I can't seem to isolate.
>
> Since I am not seeing this type of memory consumption with my
> large-file/VOD cache servers,
> and the connection profile is much different, I am guessing what may be at
> issue is the long living
> connections. ATS active timeout is currently set to 3600. This is live
> traffic so connections stay open for a hour
> and then closed.  Where as with my large-file/VOD cache servers,
> connections only remain
> open for as long as a file/title is being retrieved. 10-15mins max.
>
> Has anyone seen any issues with this type of connection profile and ATS
> 6.2.1?
>
> This doesnt seem to be an issue with 7.0, so not sure what may have
> changed between releases.
>
> Thanks!
>
>
>
> On Tue, Jun 6, 2017 at 8:32 AM, Jeremy Payne <jp557198@gmail.com> wrote:
>
> ATS versions - 6.2.1 and 7.0.0
> Cache disk - 18G ramdisk - /dev/ram0
> System memory - 32G
> Traffic type - HLS Live
>
> While testing live HLS traffic, I noticed that ATS 6.2.1. continued to use
> ramdisk until
> eventually traffic_server was restarted(via trafic_cop) as result of 'oom
> killer'
>
> Testing with ATS 7.0.0 I saw memory use remain stable once the ramdisk
> reached it's
> 'configured' capacity.
>
> I looked through the 7.0 changelog and didnt see anything obvious, so
> maybe someone is aware of an undocumented change which impacts ATS
> 'honoring' configured  ramdisk boundaries.
> Understood ATS may not know the difference between /dev/sdX and
> /dev/ramX.. But just
> putting this out to the community just in case I am missing something.
>
> Jeremy
>
>
>

Mime
View raw message