Linux Performance Monotor

When the system is running fast/normally you need to take a baseline. Look at top, see what your percentage system, user, idle and iowait is. These can be seen here:

Cpu(s):  8.6% us,  0.3% sy,  0.0% ni, 90.5% id,  0.6% wa,  0.0% hi, 0.0% si

us
    percent cpu being used by userland code
sy
    percent cpu being used by kernelspace code
ni
    like "us" but related to "niced" processes
id
    idle
wa
    cpu is idle because it waits for IO to complete
hi
    interrupts generated by hardware
si
    interrupts generated by software

vmstat 5

r 
    processes ready to run but are waiting for the cpu
b 
    processes in uninterruptible sleep (often io-related)

swpd 
    swap used (KiB)
free 
    memory doing nothing (KiB)
buff 
    buffers (KiB)
cache
    disk cache (KiB)

si 
    memory swapped in from disk (KiB/s)
so 
    memory swapped out from disk (KiB/s)

bi 
    Blocks read in from block devices (blocks/sec)
bo 
    Blocks written out to block devices (blocks/sec)

in 
    interrupts per second (hardware, software, e.g. "the clock ticked" "the ethernet card got a packet")
cs 
    context switches per second (one process stops running on CPU, another starts)

us 
    Userland CPU usage %
sy 
    Kernelspace CPU usage %
id 
    Idle CPU %
wa 
    IO wait CPU %


http://strugglers.net/wiki/Linux_performance_tuning

The first line in top:

top - 22:09:08 up 14 min,  1 user,  load average: 0.21, 0.23, 0.30

“22:09:08″ is the current time; “up 14 min” shows how long the system has been up for; “1 user” how many users are logged in; “load average: 0.21, 0.23, 0.30″ the load average of the system (1minute, 5 minutes, 15 minutes).

Load average is an extensive topic and to understand its inner workings can be daunting. The simplest of definitions states that load average is the cpu utilization over a period of time. A load average of 1 means your cpu is being fully utilized and processes are not having to wait to use a CPU. A load average above 1 indicates that processes need to wait and your system will be less responsive. If your load average is consistently above 3 and your system is running slow you may want to upgrade to more CPU’s or a faster CPU.

The second line in top:

Tasks:  82 total,   1 running,  81 sleeping,   0 stopped,   0 zombie

Shows the number of processes and their current state.

The third lin in top:

Cpu(s):  9.5%us, 31.2%sy,  0.0%ni, 27.0%id,  7.6%wa,  1.0%hi, 23.7%si,  0.0%st

Shows CPU utilization details. “9.5%us” user processes are using 9.5%; “31.2%sy” system processes are using 31.2%; “27.0%id” percentage of available cpu; “7.6%wa” time CPU is waiting for IO.

When first analyzing the Cpu(s) line in top look at the %id to see how much cpu is available. If %id is low then focus on %us, %sy, and %wa to determine what is using the CPU.

The fourth and fifth lines in top:

Mem:    255592k total,   167568k used,    88024k free,    25068k buffers
Swap:   524280k total,        0k used,   524280k free,    85724k cached

Describes the memory usage. These numbers can be misleading. “255592k total” is total memory in the system; “167568K used” is the part of the RAM that currently contains information; “88024k free” is the part of RAM that contains no information; “25068K buffers and 85724k cached” is the buffered and cached data for IO.

So what is the actual amount of free RAM available for programs to use ?

The answer is: free + (buffers + cached)

88024k + (25068k + 85724k) = 198816k

How much RAM is being used by progams ?

The answer is: used – (buffers + cached)

167568k – (25068k + 85724k) = 56776k

The processes information:

Top will display the process using the most CPU usage in descending order. Lets describe each column that represents a process.

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
3166 apache    15   0 29444 6112 1524 S  6.6  2.4   0:00.79 httpd

PID – process ID of the process

USER – User who is running the process

PR – The priority of the process

NI – Nice value of the process (higher value indicates lower priority)

VIRT – The total amount of virtual memory used

RES – Resident task size

SHR – Amount of shared memory used

S – State of the task. Values are S (sleeping), D (uninterruptible sleep), R (running), Z (zombies), or T (stopped or traced)

%CPU – Percentage of CPU used

%MEM – Percentage of Memory used

TIME+ – Total CPU time used

COMMAND – Command issued
http://www.kernelhardware.org/linux-top-command/