trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vamsi Ambati <va...@neumob.com>
Subject Re: Address Sanitizer issue in detecting memory leak.
Date Wed, 16 Nov 2016 19:12:49 GMT
Address Sanitizer issue is resolved at my end. I am providing the details how it was solved
, may be helpful for others.

Address sanitizer doesn’t detect  memory leaks in Ubuntu 14.04 LTS with gcc version 4.8.4
.
I have found a patch to gcc for Address Sanitizer  at https://gcc.gnu.org/ml/gcc-patches/2013-11/msg01874.html.
I have found the code changes related to this patch in the official gcc 4.9.X release.
When I upgraded to gcc 4.9, then address sanitizer started detecting memory leaks .
One more observation is that Address sanitizer throws  the memory leak log only when program
exits , if it is running in a for ever loop then it doesn’t 
show the leak log.

Regards
   Vamsi


> On Nov 15, 2016, at 11:57 PM, Bryan Call <bcall@apache.org> wrote:
> 
> Don’t know why Apple Mail didn’t send this.  I noticed after restarting:
> 
> By default you should see some leaks from CRYPTO_malloc even if you don’t have TLS/SSL
configured.  I am running Fedora 24.
> 
> [bcall@homer trafficserver]$ sudo /usr/local/bin/traffic_server
> traffic_server: using root directory '/usr/local'
> ^Ctraffic_server: Interrupt (Signal sent by the kernel 0 0)
> 
> =================================================================
> ==9081==ERROR: LeakSanitizer: detected memory leaks
> 
> Direct leak of 48 byte(s) in 1 object(s) allocated from:
>     #0 0x7f2a242f7e60 in malloc (/lib64/libasan.so.3+0xc6e60)
>     #1 0x7f2a231320b7 in CRYPTO_malloc (/lib64/libcrypto.so.10+0x6e0b7)
> 
> Indirect leak of 120 byte(s) in 1 object(s) allocated from:
>     #0 0x7f2a242f7e60 in malloc (/lib64/libasan.so.3+0xc6e60)
>     #1 0x7f2a231320b7 in CRYPTO_malloc (/lib64/libcrypto.so.10+0x6e0b7)
> 
> SUMMARY: AddressSanitizer: 168 byte(s) leaked in 2 allocation(s).
> 
> -Bryan
> 
> 
> 
> 
>> On Nov 10, 2016, at 2:22 PM, Vamsi Ambati <vamsi@neumob.com <mailto:vamsi@neumob.com>>
wrote:
>> 
>> Hi Sudheer,
>> 
>> Thanks for the quick response.
>> I tried with your suggested code snippet but Address sanitizer doesn’t detect the
leak.
>> I even tried with gdb by introducing break point at __asan_report_error.
>> 
>> 
>> Vamsi
>> 
>>> On Nov 10, 2016, at 1:58 PM, Sudheer Vinukonda <sudheervinukonda@yahoo.com
<mailto:sudheervinukonda@yahoo.com>> wrote:
>>> 
>>> Not sure if the *leak* in your test code below is detectable..since your program
is exiting after allocating the memory (system should automatically reclaim the memory on
process exit).
>>> 
>>> You may try to modify it to something more explicit like the below - 
>>> 
>>>> #include <stdlib.h>
>>>> void dummy() {
>>>>   char *c = (char*) malloc(20);
>>>    c = (char*) malloc(100);
>>>> }
>>>> int main() {
>>>>   dummy();
>>>>   while(1) {}
>>>> }
>>> 
>>> 
>>> 
>>> On Nov 10, 2016, at 1:43 PM, Vamsi Ambati <vamsi@neumob.com <mailto:vamsi@neumob.com>>
wrote:
>>> 
>>>> 
>>>> I am trying to resolve a memory leak issue with ATS 7.0 version.
>>>> We are using ATS from the CDN perspective and developed 3 plugins.
>>>> DevOps complains that  traffic_server process gradually consumes lot of memory
and does release back which is a sign of memory leak.
>>>> I tried with Valgrind but it didn’t work because of LUA and one of our
ATS community member suggested to  use Address Sanitizer.
>>>> I followed exactly the steps mentioned   Debugging Traffic Server using ASAN
<https://cwiki.apache.org/confluence/download/attachments/56066455/summit_asan.pptx?version=1&modificationDate=1429916307000&api=v2>.
>>>> 
>>>> I have build the traffic server with CXXFLAGS=-fno-omit-frame-pointer -fsanitize=address
>>>> Verified that traffic_server is build with ASAN library by  'ldd bin/traffic_server’
>>>> Restarted the traffic server with 'service nm-trafficserver restart'.
>>>> 
>>>> I have adopted three approaches to catch the memory leak
>>>> 
>>>> Approach 1: 
>>>> Just running the traffic server(with -f option) and continuously sending
HTTP requests using wrk  (from another machine) by attaching 
>>>> to a gdb .
>>>> ' gdb $(pidof traffic_server)’
>>>> break __asan_report_error.
>>>> 
>>>> Issue: I didn’t hit this function from the memory leak perspective.
>>>> 
>>>> Approach 2:
>>>> I have run the traffic server with PROXY_AUTO_EXIT and sending the http requests

>>>> PROXY_AUTO_EXIT=30 ASAN_OPTIONS=detect_leaks=1:verbosity=2  bin/traffic_server
-f
>>>> 
>>>> Issue: Tool didn’t complain any thing about the  memory leak.
>>>> 
>>>> 
>>>> Approach 3:
>>>> I have created simple code explicitly introducing the memory leak
>>>> 
>>>> #include <stdlib.h>
>>>> void dummy() {
>>>>   malloc(20);
>>>> }
>>>> int main() {
>>>>   dummy();
>>>>   return 0;
>>>> }
>>>> 
>>>> Compiled with g++ -fsanitize=address add.cpp 
>>>> Executed like ASAN_OPTIONS=detect_leaks=1 ./a.out
>>>> 
>>>> Issue: Tool didn’t detect this memory leak.
>>>> 
>>>> 
>>>> Production System Configuration:
>>>> 
>>>> 16Gb Ram with 8 cores VM.
>>>> gcc --version
>>>> 	gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
>>>> 
>>>> Did any one encounter this  ASAN  issue on Ubuntu 14.04  ? If so, how did
they resolve it ?
>>>> Any other suggestions how to narrow down the memory leak issue with ATS ?
>>>> 
>>>> 
>>>> 
>>>> Regards
>>>>    Vamsi
>>>> 
>>>> PS: I am new to the community and new to Apache traffic server open source
module too.
>>>> 
>>>> 
>>>> 
>>>> 
>> 
> 


Mime
View raw message