trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vamsi Ambati <va...@neumob.com>
Subject Address Sanitizer issue in detecting memory leak.
Date Thu, 10 Nov 2016 21:43:52 GMT

I am trying to resolve a memory leak issue with ATS 7.0 version.
We are using ATS from the CDN perspective and developed 3 plugins.
DevOps complains that  traffic_server process gradually consumes lot of memory and does release
back which is a sign of memory leak.
I tried with Valgrind but it didn’t work because of LUA and one of our ATS community member
suggested to  use Address Sanitizer.
I followed exactly the steps mentioned   Debugging Traffic Server using ASAN <https://cwiki.apache.org/confluence/download/attachments/56066455/summit_asan.pptx?version=1&modificationDate=1429916307000&api=v2>.

I have build the traffic server with CXXFLAGS=-fno-omit-frame-pointer -fsanitize=address
Verified that traffic_server is build with ASAN library by  'ldd bin/traffic_server’
Restarted the traffic server with 'service nm-trafficserver restart'.

I have adopted three approaches to catch the memory leak

Approach 1: 
Just running the traffic server(with -f option) and continuously sending HTTP requests using
wrk  (from another machine) by attaching 
to a gdb .
' gdb $(pidof traffic_server)’
break __asan_report_error.

Issue: I didn’t hit this function from the memory leak perspective.

Approach 2:
I have run the traffic server with PROXY_AUTO_EXIT and sending the http requests 
PROXY_AUTO_EXIT=30 ASAN_OPTIONS=detect_leaks=1:verbosity=2  bin/traffic_server -f

Issue: Tool didn’t complain any thing about the  memory leak.


Approach 3:
I have created simple code explicitly introducing the memory leak

#include <stdlib.h>
void dummy() {
  malloc(20);
}
int main() {
  dummy();
  return 0;
}

Compiled with g++ -fsanitize=address add.cpp 
Executed like ASAN_OPTIONS=detect_leaks=1 ./a.out

Issue: Tool didn’t detect this memory leak.


Production System Configuration:

16Gb Ram with 8 cores VM.
gcc --version
	gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

Did any one encounter this  ASAN  issue on Ubuntu 14.04  ? If so, how did they resolve it
?
Any other suggestions how to narrow down the memory leak issue with ATS ?



Regards
   Vamsi

PS: I am new to the community and new to Apache traffic server open source module too.





Mime
View raw message