The dbx and libumem utilities are pretty slow in finding leaks. in my experience I didnt get callstacks for memory leaks. so I used Dtrace tool which can print malloc/free callstacks and size to outputfile. From the output file we can extract malloc addresses which are not present in free ptr address. these addresses and callstacks will be memory leaked ones.
Step 1 ) use this dtrace script to print malloc/free statistics . usage is ./memleak.d pid > output.txt
#!/usr/sbin/dtrace -s
#echo " usage ./memleak.d pid "
pid$1:libc.so.1:malloc:entry
{
self->trace = 1;
self->size = arg0;
}
pid$1:libc.so.1:malloc:return
/self->trace == 1/
{
printf("Ptr=0x%p Size=%d", arg1, self->size);
ustack();
self->trace = 0;
self->size = 0;
}
pid$1:libc.so.1:free:entry
{
printf("Ptr=0x%p ", arg0);
}
for example if the process id is 1234 then
/memleak.d 1234 > output.txt
step 2 )
copy this python script to showleaks.py and run python ./showleaks.py output.txt 6 > leaks.txt
output.txt is the output file you got from dtrace
6 is the depth of callstack. make sure your output.txt has minimum of callstack depth 6.
#showleaks.py
import sys
import re
if (len(sys.argv) > 1 ):
allocsfile = open(sys.argv[1],'r')
allocs_lines=list(allocsfile.readlines())
linenums=len(allocs_lines)
i=0
poin_size=" "
''' str1='12 66201 malloc:return Ptr=0x10244cb30 Size=73' '''
allocs_map={}
stacks_map={}
final_map={}
depth=int(sys.argv[2])
malloc_set=set(" ")
free_set=set(" ")
while i < linenums:
malloc_match=re.search(r"malloc:return Ptr=\b0[xX][0-9a-fA-F]+\b Size=[0-9]*", allocs_lines[i])
free_match=re.search(r"free:entry Ptr=\b0[xX][0-9a-fA-F]+\b",allocs_lines[i])
if malloc_match:
malloc_line=str(malloc_match.group())
pos_ptr=malloc_line.find('=')
ptr_address=str(malloc_line[pos_ptr+1:pos_ptr+12])
malloc_set.add(ptr_address)
allocs_map[ptr_address]=i # store ptr addr and line number
elif free_match:
free_line=str(free_match.group())
pos_ptr=free_line.find('=')
freeptr_address=str(free_line[pos_ptr+1::])
free_set.add(freeptr_address)
i=i+1
else:
print("usage: ./showleaks.py")
leak_set=malloc_set - free_set
for i in leak_set:
j=allocs_map[i]
formatstr="malloc:return Ptr=%s Size=[0-9]*" % i
malloc_match=re.search(formatstr,allocs_lines[j])
if malloc_match:
malloc_line=str(malloc_match.group())
pos_ptr=malloc_line.find('=')
rest_of_line=str(malloc_line[pos_ptr+1::])
pos_Size=rest_of_line.find('=')
Size = int(rest_of_line[pos_Size+1::])
callstack=str(allocs_lines[j+1:j+depth])
if callstack in stacks_map:
value=stacks_map[callstack]
stacks_map[callstack] = value + Size
else:
stacks_map[callstack] = Size
keys=stacks_map.keys()
for k in keys:
print("leaked bytes: %d" %stacks_map[k])
stacks=str(k)
stacks1=stacks.split(r'\n')
for sts in stacks1:
print(sts)
print("\n DONE ")
for any queries, pls do post here. I will respond to it .
Step 1 ) use this dtrace script to print malloc/free statistics . usage is ./memleak.d pid > output.txt
#!/usr/sbin/dtrace -s
#echo " usage ./memleak.d pid "
pid$1:libc.so.1:malloc:entry
{
self->trace = 1;
self->size = arg0;
}
pid$1:libc.so.1:malloc:return
/self->trace == 1/
{
printf("Ptr=0x%p Size=%d", arg1, self->size);
ustack();
self->trace = 0;
self->size = 0;
}
pid$1:libc.so.1:free:entry
{
printf("Ptr=0x%p ", arg0);
}
for example if the process id is 1234 then
/memleak.d 1234 > output.txt
step 2 )
copy this python script to showleaks.py and run python ./showleaks.py output.txt 6 > leaks.txt
output.txt is the output file you got from dtrace
6 is the depth of callstack. make sure your output.txt has minimum of callstack depth 6.
#showleaks.py
import sys
import re
if (len(sys.argv) > 1 ):
allocsfile = open(sys.argv[1],'r')
allocs_lines=list(allocsfile.readlines())
linenums=len(allocs_lines)
i=0
poin_size=" "
''' str1='12 66201 malloc:return Ptr=0x10244cb30 Size=73' '''
allocs_map={}
stacks_map={}
final_map={}
depth=int(sys.argv[2])
malloc_set=set(" ")
free_set=set(" ")
while i < linenums:
malloc_match=re.search(r"malloc:return Ptr=\b0[xX][0-9a-fA-F]+\b Size=[0-9]*", allocs_lines[i])
free_match=re.search(r"free:entry Ptr=\b0[xX][0-9a-fA-F]+\b",allocs_lines[i])
if malloc_match:
malloc_line=str(malloc_match.group())
pos_ptr=malloc_line.find('=')
ptr_address=str(malloc_line[pos_ptr+1:pos_ptr+12])
malloc_set.add(ptr_address)
allocs_map[ptr_address]=i # store ptr addr and line number
elif free_match:
free_line=str(free_match.group())
pos_ptr=free_line.find('=')
freeptr_address=str(free_line[pos_ptr+1::])
free_set.add(freeptr_address)
i=i+1
else:
print("usage: ./showleaks.py
leak_set=malloc_set - free_set
for i in leak_set:
j=allocs_map[i]
formatstr="malloc:return Ptr=%s Size=[0-9]*" % i
malloc_match=re.search(formatstr,allocs_lines[j])
if malloc_match:
malloc_line=str(malloc_match.group())
pos_ptr=malloc_line.find('=')
rest_of_line=str(malloc_line[pos_ptr+1::])
pos_Size=rest_of_line.find('=')
Size = int(rest_of_line[pos_Size+1::])
callstack=str(allocs_lines[j+1:j+depth])
if callstack in stacks_map:
value=stacks_map[callstack]
stacks_map[callstack] = value + Size
else:
stacks_map[callstack] = Size
keys=stacks_map.keys()
for k in keys:
print("leaked bytes: %d" %stacks_map[k])
stacks=str(k)
stacks1=stacks.split(r'\n')
for sts in stacks1:
print(sts)
print("\n DONE ")
for any queries, pls do post here. I will respond to it .
9 comments:
I have followed the steps mentioned above but iam getting a error
dtrace: failed to initialize dtrace: DTrace requires additional privileges
we should use dtrace as root user.
running the second script doesn't provide any input... its just print DONE in output file.
But I can see in my memory graph for that process that there is memory leak...
Can you please elaborate the working of second script?
did u give dtrace output as input to python script?.
u can also email ur dtrace output if u want to smart.ram856@gmail.com i can run python script and give output
I have followed the steps, but the first script memleak.d is not providing any output.
The output file generated is blank.
Please advice.
u need to run memleak.d script as root user and try. also give pid as argument.
did u give the output of dtrace to python script ?. just email me your dtrace output if you can. i will look into that . this script is being used in oracle for finding leaks.
my email: smart.ram856@gmail.com
the scripts worked, great job. now I just need understand what actually happened :)
Post a Comment