Solaris - IO benchmarking - vdbench

do you want to measure performance of disks on solaris ?. then we have excellent tool called vdbench.

Initially I used to write c++ program which read/writes chunks of data and measure the Input/Output using iostat command. then i came across called vdbench which simulates read/write on given partition of hard disk .

go to http://sourceforge.net/projects/vdbench/ to download the tool and unzip it. 

1) create a file of 100 MB . $mkfile 100m /benchmark/file1 

2)now lets generate read/writes on this file /benchmark/file1 which is mounted on partition you want to benchmark IO. 

there are example files that come with installation like example1, example2 etc. we can edit example1 file ,

*Example 1: Single run, one raw disk

*SD:    Storage Definition

*WD:    Workload Definition

*RD:    Run Definition

*

sd=sd1,lun=/benchmark/file1

wd=wd1,sd=sd1,xfersize=4194304,rdpct=50

rd=run1,wd=wd1,iorate=100,elapsed=10,interval=1

* lun specifies the file, rawdisks etc.

*rdpct means read percentage. here i gave rdpct=50 means read % is 50, write % is 50. 

*xfersize is 4194304 bytes which means vdbench generates 4 MB of load. 

*elapsed is how many samples we need . we get 10 samples here 

3) issue the command $ vdbench -f example1 

Apr 20, 2012  interval        i/o   MB/sec   bytes   read     resp     resp     resp    cpu%  cpu%
                             rate  1024**2     i/o    pct     time      max   stddev sys+usr   sys
07:25:53.061         1      92.00  1559.74 4194304  53.26   13.379   26.133    2.459     2.1   2.0
07:25:54.021         2     108.00  1831.00 4194304  50.00   12.848   16.296    1.704     2.5   2.3
07:25:55.019         3     114.00  1932.72 4194304  54.39   12.824   24.141    2.158     2.5   2.4
07:25:56.018         4      93.00  1576.69 4194304  46.24   12.893   27.468    2.312     2.1   2.0
07:25:57.017         5     103.00  1746.23 4194304  52.43   12.491   15.294    1.514     2.2   2.1
07:25:58.016         6     118.00  2000.53 4194304  43.22   13.324   30.540    2.848     2.7   2.5
07:25:59.018         7     103.00  1746.23 4194304  44.66   12.661   16.723    1.527     2.2   2.1
07:26:00.016         8     102.00  1729.27 4194304  44.12   12.657   15.030    1.419     2.2   2.1
07:26:01.016         9     116.00  1966.63 4194304  45.69   12.635   16.636    1.483     2.5   2.5
07:26:02.019        10      90.00  1525.83 4194304  47.78   12.499   14.357    1.419     1.9   1.8
07:26:02.026  avg_2-10     105.22  1783.90 4194304  47.62   12.770   30.540    1.908     2.3   2.2
07:26:02.489 Vdbench execution completed successfully

the last line avg_2-10 is avg of all samples.

for 4mb read/writes here the response time is 12 secs. cpu usage is 2.3%

Memory leaks - Linux , Windows , Solaris

Debug Memory leaks on Linux using valgrind :
http://techie-builder.blogspot.com/2007/12/valgrind-memory-leak-detector.html

Debug Memory leaks on Windows using UMDH.exe :
http://techie-builder.blogspot.com/2010/12/debug-memory-leaks-c-apps-windows.html

Debug Memory leaks on Solaris 10 using Dtrace:
http://techie-builder.blogspot.com/2012/01/dtrace-to-find-memory-leaks-solaris-10.html

Dtrace to find memory leaks - Solaris 10

The dbx and libumem utilities are pretty slow in finding leaks. in my experience I didnt get callstacks for memory  leaks.  so I used Dtrace tool which can print malloc/free callstacks and size to outputfile. From the output file we can extract malloc addresses which are not present in free ptr address. these addresses and callstacks will be memory leaked ones.

Step 1 ) use this dtrace script to print malloc/free statistics . usage is ./memleak.d pid  > output.txt

#!/usr/sbin/dtrace -s
#echo " usage ./memleak.d pid "
pid$1:libc.so.1:malloc:entry
{
        self->trace = 1;
        self->size = arg0;
}
pid$1:libc.so.1:malloc:return
/self->trace == 1/
{
        printf("Ptr=0x%p Size=%d", arg1, self->size);
        ustack();
        self->trace = 0;
        self->size = 0;
}

pid$1:libc.so.1:free:entry
{
        printf("Ptr=0x%p ", arg0);
}

for example if the process id is 1234 then
/memleak.d  1234 > output.txt

 step 2 )
copy this python script to showleaks.py and run  python ./showleaks.py output.txt 6 > leaks.txt 
output.txt is the output file  you got from dtrace 
6 is the depth of callstack. make sure your output.txt has minimum of callstack depth 6. 

#showleaks.py


import sys
import re

if (len(sys.argv) > 1 ):
    allocsfile = open(sys.argv[1],'r')
    allocs_lines=list(allocsfile.readlines())
    linenums=len(allocs_lines)
    i=0
    poin_size=" "
    ''' str1='12  66201                    malloc:return Ptr=0x10244cb30 Size=73' '''
    allocs_map={}
    stacks_map={}
    final_map={}
    depth=int(sys.argv[2])

    malloc_set=set(" ")
    free_set=set(" ")
    while i < linenums:
   
        malloc_match=re.search(r"malloc:return Ptr=\b0[xX][0-9a-fA-F]+\b Size=[0-9]*", allocs_lines[i])
        free_match=re.search(r"free:entry Ptr=\b0[xX][0-9a-fA-F]+\b",allocs_lines[i])
        if malloc_match:
            malloc_line=str(malloc_match.group())
            pos_ptr=malloc_line.find('=')
            ptr_address=str(malloc_line[pos_ptr+1:pos_ptr+12])
            malloc_set.add(ptr_address)
            allocs_map[ptr_address]=i # store ptr addr and line number
        elif free_match:
            free_line=str(free_match.group())
            pos_ptr=free_line.find('=')
            freeptr_address=str(free_line[pos_ptr+1::])  
            free_set.add(freeptr_address)
        i=i+1
else:
    print("usage: ./showleaks.py ")

leak_set=malloc_set - free_set


for i in leak_set:
        j=allocs_map[i]
        formatstr="malloc:return Ptr=%s Size=[0-9]*" % i
        malloc_match=re.search(formatstr,allocs_lines[j])
        if malloc_match:
            malloc_line=str(malloc_match.group())
            pos_ptr=malloc_line.find('=')
            rest_of_line=str(malloc_line[pos_ptr+1::])
            pos_Size=rest_of_line.find('=')
            Size = int(rest_of_line[pos_Size+1::])
            callstack=str(allocs_lines[j+1:j+depth])

            if callstack in stacks_map:
                value=stacks_map[callstack]
                stacks_map[callstack] = value + Size
            else:
                stacks_map[callstack] = Size

keys=stacks_map.keys()

for k in keys:
    print("leaked bytes: %d" %stacks_map[k])
    stacks=str(k)
    stacks1=stacks.split(r'\n')
    for sts in stacks1:
        print(sts)
   
print("\n DONE ")
   
   
   

for any queries, pls do post here. I will respond to it .