|
Introduction to iostat , vmstat and netstat
This document is primarily written with reference to solaris performance monitoring and tuning but these tools are available in other unix variants also with slight syntax difference.
iostat , vmstat and netstat are three most commonly used tools for performance monitoring . These comes built in with the operating system and are easy to use .iostat stands for input output statistics and reports statistics for i/o devices such as disk drives . vmstat gives the statistics for virtual Memory and netstat gives the network statstics .
Following paragraphs describes these tools and their usage for performance monitoring and if you need more information there are some very good solaris performance monitoring books available at www.besttechbooks.com.
Table of content :
1. Iostat
2. vmstat
3. netstat
4. Next Steps
Input Output statistics ( iostat )
iostat reports terminal and disk I/O activity and CPU utilization. The first line of output is for the time period since boot & each subsequent line is for the prior interval . Kernel maintains a number of counters to keep track of the values.
iostat's activity class options default to tdc (terminal, disk, and CPU). If any other option/s are specified, this default is completely overridden i.e. iostat -d will report only statistics about the disks.
syntax:
Basic synctax is iostat <options> interval count
option - let you specify the device for which information is needed like disk , cpu or terminal. (-d , -c , -t or -tdc ) . x options gives the extended statistics .
interval - is time period in seconds between two samples . iostat 4 will give data at each 4 seconds interval.
count - is the number of times the data is needed . iostat 4 5 will give data at 4 seconds interval 5 times
|
|
|
|
Example
$ iostat -xtc 5 2
extended disk statistics tty cpu
disk r/s w/s Kr/s Kw/s wait actv svc_t %w %b tin tout us sy wt id
sd0 2.6 3.0 20.7 22.7 0.1 0.2 59.2 6 19 0 84 3 85 11 0
sd1 4.2 1.0 33.5 8.0 0.0 0.2 47.2 2 23
sd2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
sd3 10.2 1.6 51.4 12.8 0.1 0.3 31.2 3 31
The fields have the following meanings:
disk name of the disk
r/s reads per second
w/s writes per second
Kr/s kilobytes read per second
Kw/s kilobytes written per second
wait average number of transactions waiting for service (Q length)
actv average number of transactions actively
being serviced (removed from the
queue but not yet
completed)
%w percent of time there are transactions waiting
for service (queue non-empty)
%b percent of time the disk is busy (transactions
in progress)
|
Results and Solutions:
The values to look from the iostat output are:
- Reads/writes per second (r/s , w/s)
- Percentage busy (
%b)
- Service time (
svc_t)
If a disk shows consistently high reads/writes along with , the percentage busy (%b) of the disks is greater than 5 percent, and the average service time (svc_t) is greater than 30 milliseconds, then one of the following action needs to be taken
1.)Tune the application to use disk i/o more efficiently by modifying the disk queries and using available cache facilities of application servers .
2.) Spread the file system of the disk on to two or more disk using disk striping feature of volume manager /disksuite etc.
3.) Increase the system parameter values for inode cache , ufs_ninode , which is Number of inodes to be held in memory. Inodes are cached globally (for UFS), not on a per-file system basis
4.) Move the file system to another faster disk /controller or replace existing disk/controller to a faster
one.
Virtual Memory Statistics ( vmstat )
vmstat - vmstat reports virtual memory statistics of process, virtual memory, disk, trap, and CPU activity.
On multicpu systems , vmstat averages the number of CPUs into the output. For per-process statistics .Without options, vmstat displays a one-line summary of the virtual memory activity since the system was booted.
syntax:
Basic synctax is vmstat <options> interval count
option - let you specify the type of information needed such as paging -p , cache -c ,.interrupt -i etc.
if no option is specified information about process , memory , paging , disk ,interrupts & cpu is displayed .
interval - is time period in seconds between two samples . vmstat 4 will give data at each 4 seconds interval.
count - is the number of times the data is needed . vmstat 4 5 will give data at 4 seconds interval 5
times.
Example
The following command displays a summary of what the system
is doing every five seconds.
example% vmstat 5
procs memory page disk faults cpu
r b w swap free re mf pi p fr de sr s0 s1 s2 s3 in sy cs us sy id
0 0 0 11456 4120 1 41 19 1 3 0 2 0 4 0 0 48 112 130 4 14 82
0 0 1 10132 4280 0 4 44 0 0 0 0 0 23 0 0 211 230 144 3 35 62
0 0 1 10132 4616 0 0 20 0 0 0 0 0 19 0 0 150 172 146 3 33 64
0 0 1 10132 5292 0 0 9 0 0 0 0 0 21 0 0 165 105 130 1 21 78
|
The fields of vmstat's display are
procs
r in run queue
b blocked for resources I/O, paging etc.
w swapped
memory (in Kbytes)
swap - amount of swap space currently available
free - size of the free list
page ( in units per second).
re page reclaims - see -S option for how this field is modified.
mf minor faults - see -S option for how this field is modified.
pi kilobytes paged in
po kilobytes paged out
fr kilobytes freed
de anticipated short-term memory shortfall (Kbytes)
sr pages scanned by clock algorithm
disk ( operations per second )
There are slots for up to four disks, labeled with a single letter and number.
The letter indicates the type of disk (s = SCSI, i = IPI, etc) . The number is
the logical unit number.
faults
in (non clock) device interrupts
sy system calls
cs CPU context switches
cpu - breakdown of percentage usage of CPU time. On multiprocessors this is an a
verage across all processors.
us user time
sy system time
id idle time
Results and Solutions:
A. CPU issues:
Following columns has to be watched to determine if there is any cpu issue
- Processes in the run queue (procs r)
- User time (cpu us)
- System time (cpu sy)
- Idle time (cpu id)
procs cpu
r b w us sy id
0 0 0 4 14 82
0 0 1 3 35 62
0 0 1 3 33 64
0 0 1 1 21 78
|
Problem symptoms:
1.) If the number of processes in run queue (procs r) are consistently greater than the number of CPUs on the system it will slow down system as there are more processes then available CPUs .
2.) if this number is more than four times the number of available CPUs in the system then system is facing shortage of cpu power and will greatly slow down the processess on the system.
3.) If the idle time (cpu id) is consistently 0 and if the system time (cpu sy) is double the user time (cpu us) system is facing shortage of CPU resources.
Resolution :
Resolution to these kind of issues involves tuning of application procedures to make efficient use of cpu and as a last resort increasing the cpu power or adding more cpu to the system.
B. Memory Issues:
Memory bottlenecks are determined by the scan rate (sr) . The scan rate is the pages scanned by the clock algorithm per second. If the scan rate (sr) is continuously over 200 pages per second then there is a memory shortage.
Resolution :
1. Tune the applications & servers to make efficient use of memory and cache.
2. Increase system memory .
3. Implement priority paging in s in pre solaris 8 versions by adding line "set priority paging=1" in
/etc/system. Remove this line if upgrading from Solaris 7 to 8 & retaining old /etc/system file.
Network Statistics (netstat)
netstat displays the contents of various network-related data structures in depending on the options selected.
Syntax :
netstat <option/s>
multiple options can be given at one time.
Options
-a - displays the state of all sockets.
-r - shows the system routing tables
-i - gives statistics on a per-interface basis.
-m - displays information from the network memory buffers. On Solaris, this shows statistics
forSTREAMS
-p [proto] - retrieves statistics for the specified protocol
-s - shows per-protocol statistics. (some implementations allow -ss to remove fileds with a value of 0 (zero) from the display.)
-D - display the status of DHCP configured interfaces.
-n do not lookup hostnames, display only IP addresses.
-d (with -i) displays dropped packets per interface.
-I [interface] retrieve information about only the specified interface.
-v be verbose
interval - number for continuous display of statictics.
Example :
$netstat -rn
Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ------ ---------
192.168.1.0 192.168.1.11 U 1 1444 le0
224.0.0.0 192.168.1.11 U 1 0 le0
default 192.168.1.1 UG 1 68276
127.0.0.1 127.0.0.1 UH 1 10497 lo0 |
This shows the output on a Solaris machine who's IP address is 192.168.1.11 with a default router at 192.168.1.1
Results and Solutions:
A.) Network availability
The command as above is mostly useful in troubleshooting network accessibility issues . When outside network is not accessible from a machine check the following
1. if the default router ip address is correct
2. you can ping it from your machine.
3. If router address is incorrect it can be changed with route add commnad . See man route for more info .
route command examples:
$route add default <hostname>
$route add 192.0.2.32 <gateway_name>
If the router address is correct but still you can't ping it there may be some network cable /hub/switch problem and you have to try and eliminate the faulty component .
B.) Network Response
$ netstat -i
Name |
Mtu |
Net/Dest |
Address |
Ipkts |
Ierrs |
Opkts |
Oerrs |
Collis |
Queue |
lo0 |
8232 |
loopback |
localhost |
77814 |
0 |
77814 |
0 |
0 |
0 |
hme0 |
1500 |
server1 |
server1 |
10658566 |
3 |
4832511 |
0 |
279257 |
0 |
This option is used to diagnose the network problems when the connectivity is there but it is slow in response .
Values to look at:
- Collisions (
Collis)
- Output packets (
Opkts)
- Input errors (
Ierrs)
- Input packets (
Ipkts)
The above values will give information to workout
i. Network collision rate as follows :
Network collision rate = Output collision counts / Output packets
Network-wide collision rate greater than 10 percent will indicate
- Overloaded network,
- Poorly configured network,
- Hardware problems.
ii. Input packet error rate as follows :
Input Packet Error Rate = Ierrs / Ipkts.
If the input error rate is high (over 0.25 percent), the host is dropping packets. Hub/switch cables etc needs to be checked for potential problems.
C. Network socket & TCP Cconnection state
Netstat gives important information about network socket and tcp state . This is very useful in
finding out the open , closed and waiting network tcp connection .
Network states returned by netstat are following :
CLOSED ---- Closed. The socket is not being used.
LISTEN ---- Listening for incoming connections.
SYN_SENT ---- Actively trying to establish connection.
SYN_RECEIVED ---- Initial synchronization of the connection under way.
ESTABLISHED ---- Connection has been established.
CLOSE_WAIT ---- Remote shut down; waiting for the socket to close.
FIN_WAIT_1 ---- Socket closed; shutting down connection.
CLOSING ---- Closed, then remote shutdown; awaiting acknowledgement.
LAST_ACK ---- Remote shut down, then closed ;awaiting acknowledgement.
FIN_WAIT_2 ---- Socket closed; waiting for shutdown from remote.
TIME_WAIT ---- Wait after close for remote shutdown retransmission.
Example:
#netstat -a
|
|
| Local Address |
Remote Address |
Swind |
Send-Q |
Rwind |
Recv-Q |
State |
| *.* |
*.* |
0 |
0 |
24576 |
0 |
IDLE |
| *.22 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.22 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.* |
*.* |
0 |
0 |
24576 |
0 |
IDLE |
| *.32771 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.4045 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.25 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.5987 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.898 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.32772 |
*.* |
0 |
0 |
24576 |
0 |
LISTEN |
| *.32775 |
padding-right: 1px; padding-left: 1px; font-weight: 400; font-size: 10pt; vertical-align: bottom; color: windowtext; padding-top: 1px;
分享到：
Global site tag (gtag.js) - Google Analytics
| |
相关推荐
8. **监控工具使用**：介绍Solaris自带的性能监控工具，如`prstat`、`iostat`、`vmstat`和`netstat`，以及如何利用它们进行性能诊断。 9. **系统调优最佳实践**：提供一套完整的调优流程和建议，帮助用户从整体上...
- **性能监控工具**：介绍常用的系统监控工具，如 top、vmstat、iostat 和 netstat 等。 - **性能数据解析**：解释如何解读这些工具提供的数据，以便快速定位潜在的问题。 - **监控策略**：探讨如何设置合理的监控...
- `netstat` 用于查看网络连接状态和统计信息，可以帮助识别网络拥塞或其他网络相关的问题。 - **2.6.2 nfsstat 命令：** - `nfsstat` 专门用于监控NFS客户端和服务端的通信情况，对于优化NFS服务的性能非常有用...
- 利用`iostat`、`vmstat`等命令分析I/O和内存状态。 - 通过`perf`工具进行高级性能分析。 ##### 2. 代码覆盖率 - **概念**：代码覆盖率是一种衡量测试质量的方法，表示测试用例执行了多少比例的源代码。 - **...
`top`和`htop`提供实时的系统资源使用情况，`vmstat`和`iostat`则用于监控系统状态，`perf`工具集提供高级性能分析。通过这些工具，我们可以识别性能瓶颈，进而优化系统配置。 4. **内存调优**：了解如何调整虚拟...
5. **Performance Monitoring and Tuning Commands** - `topas`：高级系统监视器，提供实时的系统资源使用情况视图。 - `vmstat`：虚拟内存统计，分析系统内存使用、进程状态和I/O活动。 - `iostat`：输入/输出...
"Linux_Debugging_and_Performance_Tuning_Tips_and_Techniques.rar"这个压缩包文件显然包含了一个关于这个主题的资源，如".chm"文件，这通常是一个Windows帮助文档，里面可能包含了丰富的教程和指南。 Linux调试...
【Linux性能调优 Linux Performance Tuning】是一本实用的手册，专注于解决Linux系统中的性能问题和日常调优工作。本书适合运维人员以及对Linux性能优化感兴趣的IT从业者使用。书中涵盖了一系列操作系统调优方法、...
- 如 top、vmstat 等，用于实时监控系统资源的使用情况。 - **内核级监控工具** - 如 perf、trace-cmd 等，用于监控内核活动，帮助识别性能瓶颈。 - **磁盘监控工具** - 包括 iostat、hdparm 等，用于监控磁盘 I/...
使用工具如`top`、`vmstat`、`iostat`和`netstat`来监控CPU、内存、磁盘和网络的使用情况，以便及时发现瓶颈并作出相应调整。 四、内存管理 1. ** ballooning**：内存ballooning技术允许动态调整虚拟机的内存使用...
- **网络延迟**：使用`netstat`或`tcpdump`命令监控网络状况。 #### 22. RAID5与RAID01/10的理解： **知识点解析：** - **RAID5**：采用分布式奇偶校验技术，提供容错能力。 - **RAID01**：先进行条带化再进行镜像...
DBA需要了解如何利用Linux的性能分析工具，如`vmstat`、`iostat`和`top`，结合Oracle的性能优化工具，如`expdp/impdp`（数据泵导出导入）和SQL Tuning Advisor，来提升数据库的整体性能。 总的来说，《Oracle.10g....
7. 系统监控与性能调优：系统监控工具（如vmstat, iostat, free）和性能调优工具（如nice, ionice, tuning）可以帮助用户了解系统状态并优化系统性能。 8. 高级脚本工具：对于需要进行自动化任务的用户，可以使用...