trafficserver-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Malenfant <smalenf...@gmail.com>
Subject Corrupted cache? Looping on directory cleaning.
Date Tue, 01 Nov 2016 16:35:10 GMT
We have a server which had a bad disk (out of 24). Trafficserver was
stopped and restarted before the disk was swapped (the disk has been marked
offline for a week). This is on version is 5.3.2 with some patches.

Stop, cleared the cache and restarted. Problem is gone.

Once it was restarted, I kept seeing this with "perf top" and CPU usage is
100% with load average of 1. Every request is a MISS.

Anybody seen this before?

Been going for more than 24 hours and server has been idle :

  50.74%  traffic_server       [.] dir_clean_segment

  30.65%  traffic_server       [.] dir_delete_entry


If I look a disk IO, there is nothing going on. Looks like stuck in a loop.

Enabling debug produced the following :

[Nov  1 16:06:31.127] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 1A boffset 0 b 0x2b372a76a838 p (nil) l -1

[Nov  1 16:06:31.127] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 1E boffset 0 b 0x2b372a76a838 p (nil) l -1

[Nov  1 16:06:31.127] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 2E boffset 0 b 0x2b372a76a838 p (nil) l -1

[Nov  1 16:06:31.127] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 32 boffset 0 b 0x2b372a76a838 p (nil) l -1

[Nov  1 16:06:31.127] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 3A boffset 0 b 0x2b372a76a838 p (nil) l -1

[Nov  1 16:06:31.127] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 3E boffset 0 b 0x2b372a76a838 p (nil) l -1

[Nov  1 16:06:31.127] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 42 boffset 0 b 0x2b372a76a838 p (nil) l -1

[Nov  1 16:18:31.102] Server {0x2b305e827700} DEBUG: (dir_clean) cleaning
0x2b372a76a838 tag 171 boffset 0 b 0x2b372a76a838 p (nil) l -1

Mime
View raw message