trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Veiko Kukk <veiko.k...@gmail.com>
Subject Re: Understanding ATS memory usage
Date Tue, 23 Jan 2018 08:36:45 GMT
Hi again,

During that mysterious task that happens after ~ 50-51 minutes causes
requests/responses to slow down very much, even time out.
Requests that usually take few hundred milliseconds are now taking over 30s
and timing out. This happens only during that time when memory consumption
is suddenly dropped by ATS. Happens for both bypassed urls and for hits.
ATS version is 7.1.1 and this looks like serious bug for me.

Regards,
Veiko


2017-12-19 18:28 GMT+02:00 Alan Carroll <solidwallofcode@oath.com>:

> It's a complex subject hard to put in an email. A few notes:
>
> 1) You shouldn't let an ATS box swap. That almost always ends badly.
> Adding more ram or adjusting the configuration to avoid it is better. I
> think we set swappiness to 0.
>
> 2) The cache directory takes memory independent of the ram cache. This is
> 10 bytes per directory entry. The number of directory entries is roughly
> the cache disk size divided by the average object size as set in
> records.config.
>
> 3) ATS does not use the kernel page cache for its own cache operations.
>
> 4) A larger ram cache almost always creates better performance, but the
> yield curve can differ quite a lot. What the ram cache does is enable
> cached data to be served from ram instead of disk. However, once the ram
> cache covers most of the working set, additional ram yields marginal
> benefits. E.g. putting an object fetched once a day in ram cache is better,
> but not very much.
>
> 5) I think what your'e seeing with your graph is cache directory
> synchronization to disk. To do that, ATS allocates memory for a copy of the
> cache directory, copies the directory there, then writes it out. It should
> be doing that somewhat peicemeal because a full duplicate of cache
> directory can be very large.
>
>
> On Tue, Dec 19, 2017 at 3:04 AM, Veiko Kukk <veiko.kukk@gmail.com> wrote:
>
>> Hi,
>>
>> Really nobody knows how ATS uses memory?
>>
>> Veiko
>>
>> 2017-12-12 14:44 GMT+02:00 Veiko Kukk <veiko.kukk@gmail.com>:
>>
>>> Hi,
>>>
>>> I'm confused about ATS memory configuration. I have a server with CentOS
>>> 7, ATS 7.1.1, 64GB memory and ~ 10TB disk.
>>> traffic_server process takes ~ 23GB memory with the configuration option
>>> (8GB)
>>> CONFIG proxy.config.cache.ram_cache.size INT 8589934592
>>> <(858)%20993-4592>
>>> ATS is using raw partition on HDD.
>>>
>>> * Why does it swap when there is page cache that's basically free memory
>>> that could be used before swapping. vm.swappiness is 10, i had it set to 0
>>> too, then system does not swap.
>>> * Considering ATS is using O_DIRECT with raw partitions and it's own
>>> memory management for disk cache, would that mean that ATS is not using
>>> kernel page cache at all?
>>> * Would ATS benefit  from larger RAM cache considering it has it's own
>>> disk buffer management.
>>>
>>> Also, most strange is that there are frequent memory usage drops of
>>> traffic_server process. After around 50 minutes, 10GB memory is released
>>> and immediately consumed again. Attaching screenshot.
>>>
>>> Regards,
>>> Veiko
>>>
>>>
>>
>

Mime
View raw message