Tools for Performance Evaluation in Linux

Learning how to evaluate the performance of your operating system is critical to scaling up your web services to a large number of users.

This article will discuss a few tools that will help you with evaluating the performance of your Linux server and one for evaluating your webserver

top

The Linux command top provides real-time view of the running system. The command is helpful for displaying high utilizing resources. To run the command simply type the name:

$ top

The output of the screen will look something like this:


top - 18:38:09 up  2:13,  3 users,  load average: 0.57, 0.13, 0.04
Tasks:  83 total,   2 running,  81 sleeping,   0 stopped,   0 zombie
Cpu(s): 49.7%us, 39.3%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi, 11.0%si,  0.0%st
Mem:   1019452k total,   186040k used,   833412k free,    12000k buffers
Swap:        0k total,        0k used,        0k free,    99032k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND           
 3854 ec2-user  20   0 88776 8012 4956 R 21.0  0.8   0:03.69 ab                 
 3871 apache    20   0  334m  10m 6252 S 14.3  1.1   0:00.55 httpd              
 3868 apache    20   0  334m  10m 6252 S 14.0  1.1   0:00.55 httpd              
 3869 apache    20   0  334m  10m 6252 S 14.0  1.1   0:00.55 httpd              
 3870 apache    20   0  334m  10m 6252 S 14.0  1.1   0:00.55 httpd              
 3873 apache    20   0  334m  10m 6252 S  5.7  1.1   0:00.17 httpd              
 3874 apache    20   0  334m  10m 6252 S  0.7  1.1   0:00.02 httpd              
 3875 apache    20   0  334m  10m 6252 S  0.7  1.1   0:00.02 httpd              
    1 root      20   0 19612 2416 2092 S  0.0  0.2   0:00.78 init               
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd           
    3 root      20   0     0    0    0 S  0.0  0.0   0:00.03 ksoftirqd/0        
    5 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0H       
    6 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kworker/u30:0      
    7 root      20   0     0    0    0 S  0.0  0.0   0:00.07 rcu_sched          
    8 root      20   0     0    0    0 S  0.0  0.0   0:00.00 rcu_bh             
    9 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0        
   10 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 khelper

vmstat

vmstat (virtual memory statistics) is a monitoring utility, which also provides information about block IO and CPU activity in addition to memory.

vmstat provides a number of values and will typically be called using two numerical parameters.

Example:

$ vmstat 1 5

- 1 -> the values will be re-measured and reported every second
- 5 -> the values will be reported five times and then the program will stop

The first line of the report will contain the average values since the last time the computer was rebooted. All other lines in the report will represent their respective current values. Vmstat does not need any special user rights. It can run as a normal user.

[ec2-user@ip-172-30-0-240 ~]$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 1869728  11612 112776    0    0    16     2    8   19  0  0 99  0  0	
 0  0      0 1869756  11612 112776    0    0     0     0   13   22  1  0 99  0  0	
 0  0      0 1869752  11612 112776    0    0     0     0    9   11  0  0 100  0  0	
 0  0      0 1869748  11612 112776    0    0     0     0   12   18  0  0 100  0  0	
 1  0      0 1869748  11612 112776    0    0     0     0    9    8  0  0 100  0  0	
[ec2-user@ip-172-30-0-240 ~]$

Explanation of Individual Values

(Source man vmstat):

Procs
- r: The number of processes waiting for run time.
- b: The number of processes in uninterruptible sleep.
Memory
- swpd: the amount of virtual memory used.
- free: the amount of idle memory.
- buff: the amount of memory used as buffers.
- cache: the amount of memory used as cache.
- inact: the amount of inactive memory. (-a option)
- active: the amount of active memory. (-a option)
Swap
- si: Amount of memory swapped in from disk (/s).
- so: Amount of memory swapped to disk (/s).
IO
- bi: Blocks received from a block device (blocks/s).
- bo: Blocks sent to a block device (blocks/s).
System
- in: The number of interrupts per second, including the clock.
- cs: The number of context switches per second.
CPU
- These are percentages of total CPU time.
- us: Time spent running non-kernel code. (user time, including nice time)
- sy: Time spent running kernel code. (system time)
- id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
- wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
- st: Time stolen from a virtual machine.

Apache Benchmark Tool

The Apache Benchmark ab [1] provides a convenient way to stress test your web server. The tool is included with the default installation of Apache.

An example of using ab to send 1000 requests, 10 at a time to the URL http://example.com/ is shown below:

$ ab -c 10 –n 1000 http://example.com/

The output of the tool will look similar to this:


Document Path:          /
Document Length:        25951 bytes

Concurrency Level:      10
Time taken for tests:   11.072 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      26253056 bytes
HTML transferred:       25995542 bytes
Requests per second:    90.32 [#/sec] (mean)
Time per request:       110.722 [ms] (mean)
Time per request:       11.072 [ms] (mean, across all concurrent requests)
Transfer rate:          2315.51 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       32   33   0.4     33      39
Processing:    67   77  22.5     68     141
Waiting:       34   35   0.9     35      44
Total:        100  110  22.4    101     174