OS + UNIX AIX command

AIX版必读收藏贴

http://www.loveunix.net/thread-51984-1-1.html

<AIX如何快速入门到精通>    by orian

http://bbs.loveunix.net/viewthread.php?tid=26440

# 192.168.157.116

High runqueue with idle CPU.

http://www.ibm.com/developerworks/forums/thread.jspa?threadID=413621

http://groups.google.com/group/sun-java-j2ee-j2se-j2me/msg/fb970231079d251d?hl=zh-CN&
$ oslevel -r
6100-04
$ id
uid=600(wcsuser) gid=600(wcs) groups=1(staff),701(db2igwcs)
$ vmstat -Iwt 2
System configuration: lcpu=8 mem=15744MB
   kthr            memory                         page                       faults           cpu       time 
----------- --------------------- ------------------------------------ ------------------ ----------- --------
  r   b   p        avm        fre    fi    fo    pi    po    fr     sr    in     sy    cs us sy id wa hr mi se
  2   1   0    1873243     558126     1    15     0     0     2      3   668   5784  2845  4  2 94  0 11:50:34

# vmstat 1 5
观察si和so值是否较大 , 查看b参数(等待资源的进程数)和wa参数(I/O等待中所占用的CPU时间的百分比,高于30%时,则I/O负荷较大)

# iostat 1 5
如果%util 接近100%，说明产生的I/O请求太多，I/O系统已经满负荷，该磁盘可能存在瓶颈。

如果idle小于70%，I/O的压力就比较大了，说明读取进程中有较多的wait。
$ vmstat --help
vmstat: Not a recognized flag: -
usage: vmstat [ -fsviItlwhc ] [ [ -p | -P ]  pagesize | ALL ] [ -@ Wpar Name | ALL ] [ Drives ] [ -S [ Power ]] [ Interval [ Count ] ]
$ vmstat -v
              4030464 memory pages
              3892800 lruable pages
               559656 free pages
                    2 memory pools
               993922 pinned pages
                 80.0 maxpin percentage
                  3.0 minperm percentage
                 90.0 maxperm percentage
                 40.4 numperm percentage
              1573380 file pages
                  0.0 compressed percentage
                    0 compressed pages
                 40.4 numclient percentage
                 90.0 maxclient percentage
              1573380 client pages
                    0 remote pageouts scheduled
                 4455 pending disk I/Os blocked with no pbuf
                    0 paging space I/Os blocked with no psbuf
                 2484 filesystem I/Os blocked with no fsbuf
                 1670 client filesystem I/Os blocked with no fsbuf
               312546 external pager filesystem I/Os blocked with no fsbuf
                 47.1 percentage of memory used for computational pages

1。 Aix收藏夹  —— ibm6000
http://bbs.loveunix.net/viewthread.php?tid=1842&extra=page%3D1
各种资源联接地址

2。p5 resource addres  ——  orian
http://bbs.loveunix.net/viewthread.php?tid=43271&extra=page%3D1
针对P5机器的各种资源联接地址

3。一个远程的技术支持平台  ——  shala
http://bbs.loveunix.net/viewthread.php?tid=50893&extra=page%3D1
一个比较不错的远程登录技术支持工具

4。本人有RS6000小型机，可以远程供大家练手！——  rs6k_aix

http://bbs.loveunix.net/viewthread.php?tid=28202&extra=page%3D1

可以远程登录的AIX机器

5,AIX版05年全部帖子CHM文件下载  ——  大漠

http://www.loveunix.net/html/200601/57403.html

6,提问的艺术  ——  orain

http://bbs.loveunix.net/viewthread.php?tid=71373

7,《亲历惊心动魄48小时！》-要命的数据丢失  ——  yuyang0726

http://bbs.loveunix.net/viewthread.php?tid=71821

8,《菜鸟日记》-AIX高手升级宝典  ——  orain

http://bbs.loveunix.net/viewthread.php?tid=71707

9. 性能工具下载 performance tools download：
https://www.ibm.com/developerworks/wikis/display/WikiPtype/Other+Performance+Tools

10.欢迎新手，积极参与讨论权限和水平就上升了
当然，提问最好要有艺术：提问的艺术 http://bbs.loveunix.net/viewthread.php?tid=71373&extra=page%3D1

关于权限：发东西能不能不整那么高的权限 http://bbs.loveunix.net/viewthread.php?tid=73139&extra=page%3D1

关于遭遇： 一鼻子撞上了“干什么的”！http://bbs.loveunix.net/viewthread.php?tid=72946&extra=page%3D2

十全大补：来LU必读的十帖 http://bbs.loveunix.net/viewthread.php?tid=73378&extra=page%3D1

参加IBM的认证考试不要找IBM，都是有专门的考试机构的，找IBM是绕圈子，多花钱而且不方便。IBM的工程师参加认证考试也是得到外边的考试中心的。
考IBM的认证，也不需要参加培训，你能通过认证考试中心的考试，IBM就给你发证书。

认证考试地址查询：
http://www.prometric.com.cn/aptcquery.asp?page_id=84 （IBM的考试在这里）
如何注册？

• 请联系当地考场。考场信息详情请点击此处 。
• 如果您有美金信用卡（Visa, MasterCard等），100％免费的考试券（Voucher）或优惠折扣号（Promotion Code），或者您要参加的考试是免费的，请登录www.prometric.com 自行注册考试。（注：在网站上用美金信用卡注册考试的，如需发票，需在网站自己的考试历史记录里面打印英文的receipt或请考场通过注册软件打印。）

http://www.pearsonvue.com.cn/test-centers/IT/index.asp （这个现在没有IBM的考试了）

现在考试费每个地方都不同吧！有的950 有的850 现在可以有个办法让它们都一样!
你可以去普尔文网上去注册考试，约个考试时间，用信用卡支付！http://www.prometric.com.cn/openpage2.asp?page_id=389
这个链接讲的很清楚！ 费用就是95*7=665 要是汇率底了考试费还有低！


IBM官方认证考试信息查询：http://www-03.ibm.com/certify/

认证考试报名及费用查询：http://www.register.prometric.com/Menu.asp?cookie%5Ftest=1

AIX  调出主进程号pid及相关子进程号pid

#ps -mp 5046778 -o THREAD
#Kill -3 5046778
对几台服务器用ps -mp ******（进程号） -o  THREAD >thread.txt ;kill -3 ******（进程号） 而后，在thread.txt中找到CP最高的若干进程号，转换为十六进制后，在生成的javacore中找到对应的进程的消息。

AIX WebSphere Java应用CPU 异常问题定位

// 通过 WebSphere JAVA 进程查线程号
[root@testServer ~]# ps -ef | grep java
// 得到JAVA PID数字，替换下方PID
[root@testServer ~]# top -H -p PID -b -n1> dump_high_cpu.txt
// 在这个txt文件可以看到占比比较高的线程号,将这个线程号转成16位的
[root@testServer ~]# kill -3 PID
// 这个目录下会生成一个javacore文件
[root@testServer ~]# cd /opt/IBM/WebSphere/AppServer/profiles/AppSrv01
[root@testServer ~]# ls -l *.txt
-rw-r--r-- 1 wasadmin wasadmin   3807148 Jun 20 14:52 javacore.20120620.145244.1509.0002.txt
// 然后利用占比较高的十进制线程号转换之后的十六进制数在javacore.20120620.145244.1509.0002.txt文件中找这个线程及定位相关操作实现的代码类

 

AIX中远程图形界面Xmanager无法登陆的常见处理办法

http://www.cnblogs.com/elegantok/archive/2008/12/05/1348712.html
环境 产品:rs6000,平台aix,机型all,软件版本v4 or later
问题 用户通过远程图形软件（如：Xmanager)登陆aix，用户无法登陆。如何解决该问题？
解答
1。 在主控台执行
/usr/dt/bin/dtconfig -kill （关闭cde 进程）
/usr/dt/bin/dtconfig -d （下次启动机器不进cde界面）
/usr/lib/X11/xdm/xdmconf -e （打开xdm功能）
startsrc -s xdm （启动xdm)
ps -ef | grep xdm (检查xdm进程正在运行）
ps -ef | grep dtlogin （cde 进程应该停止）
2。 编辑Xservers 文件 ，
vi /usr/lib/X11/xdm/Xservers
把最后一行的#号去掉
# :0 Local local /usr/bin/X11/X -force

AIX 调出图形界面任务管理器 AIX Performance Workbench Platform Version:1.1.0 Build id:20070501

ec7svr1/#oslevel -r

6100-04
$ ls -l /usr/bin/perfwb*
-r-xr-xr-x    1 root     sys             748 Jul 01 2007  /usr/bin/perfwb

$ perfwb
Error: Couldn't find per display information
Performance Workbench: JVM terminated. Exit code=1
/usr/java5/jre/bin/java
-Dorg.apache.xerces.xni.parser.XMLParserConfiguration=org.apache.xerces.parsers.StandardParserConfiguration
-Djava.endorsed.dirs=/opt/perfwb/plugins/org.apache.xerces_4.0.13
-Djava.library.path=/opt/perfwb/plugins/com.ibm.perfwb.procmon.aix_1.0.0/os/aix/ppc
-cp /opt/perfwb/startup.jar org.eclipse.core.launcher.Main
-os aix
-ws motif
-arch ppc
-showsplash /opt/perfwb/perfwb -showsplash 600

$ export DISPLAY=192.168.157.87:0.0

$ ls -l /usr/bin/proc*
-r-xr-xr-x    1 bin      bin            8398 Sep 15 2010  /usr/bin/proccred
-r-xr-xr-x    1 bin      bin           21708 Sep 15 2010  /usr/bin/procfiles
-r-xr-xr-x    1 bin      bin           14852 Sep 15 2010  /usr/bin/procflags
-r-xr-xr-x    1 bin      bin           11222 Sep 15 2010  /usr/bin/procldd
-r-xr-xr-x    1 bin      bin           11734 Sep 15 2010  /usr/bin/procmap
-r-xr-xr-x    1 bin      bin            8274 Sep 15 2010  /usr/bin/procrun
-r-xr-xr-x    1 bin      bin            8974 Aug 06 2010  /usr/bin/procsig
-r-xr-xr-x    1 bin      bin           23434 Sep 15 2010  /usr/bin/procstack
-r-xr-xr-x    1 bin      bin            8502 Sep 15 2010  /usr/bin/procstop
-r-xr-xr-x    1 bin      bin           33276 Sep 15 2010  /usr/bin/proctree
-r-xr-xr-x    1 bin      bin           13612 Sep 15 2010  /usr/bin/procwait
-r-xr-xr-x    1 bin      bin            9158 Sep 15 2010  /usr/bin/procwdx
$ ls -l /usr/bin/per*
-r-xr-xr-x    1 root     sys             748 Jul 01 2007  /usr/bin/perfwb
lrwxrwxrwx    1 root     system           28 May 06 2011  /usr/bin/perl -> /usr/opt/perl5/bin/perl5.8.8
lrwxrwxrwx    1 root     system           28 May 06 2011  /usr/bin/perl32 -> /usr/opt/perl5/bin/perl5.8.8
lrwxrwxrwx    1 root     system           26 May 06 2011  /usr/bin/perlbug -> /usr/opt/perl5/bin/perlbug
lrwxrwxrwx    1 root     system           25 May 06 2011  /usr/bin/perlcc -> /usr/opt/perl5/bin/perlcc
lrwxrwxrwx    1 root     system           26 May 06 2011  /usr/bin/perldoc -> /usr/opt/perl5/bin/perldoc
lrwxrwxrwx    1 root     system           26 May 06 2011  /usr/bin/perlivp -> /usr/opt/perl5/bin/perlivp

aix下的perfmon: 一个易用的性能管理工具

http://blog.chinaunix.net/space.php?uid=7761148&do=blog&id=2044648

一直没有想过命令行占绝对统治地位的aix系统中会有这样一个另类的图形管理工具, 有点想简化版本的windows资源管理器, 上图看看先:




对进程管理的功能还是蛮强大的. 可以查看线程状态, 进程树, 等等. 这个工具使用profwb命令启动.

AIX on the process priority and process control

http://tutorialsto.com/index.php/os/aix/aix-on-the-process-priority-and-process-control.html

Through tools such as kill and nice management process is very simple, but if more detail on the process of management control, then how about? aix ® in the use of multi-processor system, you can process and the thread will be allocated to specific processors to achieve this goal, but how to choose the appropriate application and organize a larger system in order to carry out the corresponding application Optimization of this? In this article, you will understand the process can be used for a variety of organizations, tools, and analysis of organizations and the theory behind the selection process and learn how to effectively set the process priority.

Introduction

AIX ® as an administrator, you should have about the process of how to deal with the basic knowledge, including research process, the priority setting process and termination process. In addition, you should know how to use hand tools to adjust the process and optimize accordingly. These tools include the new AIX 5.3 in a number of tools. System to provide effective control of the process, you must understand the definition of processes and threads, as well as the distinction between the two. This article will also be introduced ps, nice, schedtune console command as well as the process monitor (procmon), AIX Workload Manager (WLM) and other tools available. Let us start with the definition of processes and threads to start:

Process - the process is the system by the command, Shell script to start the process or other activities.

Thread - the thread is an independent control of flow, it is with other independent in the process of control-flow with the internal operation of the same address space. Kernel thread is a single sequence of control flow.

Look at this distinction another way, the process is the system used to control the use of system resources of the entity, and the thread control of the processor time actually used. Most of the system management tools still require reference to the process rather than thread. In fact, the process itself also has kernel threads, each process can have one or more kernel threads (such as multi-threaded applications). For the thread, you can be different on the system CPU to run multiple threads in order to truly make use of multi-processor computer (symmetric multi-processor, also known as SMP machine) advantage. Applications can be designed to have multiple user-level threads, and their applications or plan on the grounds of the pthread scheduling libpthread procedures. Thread of control to allow multiple applications from multiple users at the same time request the provision of services. In libpthread implementation, the user thread is located on top of virtual processors, and virtual processor itself is located on top of kernel threads. In this paper, the core aspects of the process to conduct in-depth research, and exploring the possibility of help to improve the efficiency of the entire system management tools. To help you effectively manage the environment, we will detail the well-tried a lot of Unix ® commands and AIX administrator for a new type of tool use.

Threads and SMT

By allowing different CPU to run multiple threads, but also the effective use of simultaneous multithreading (SMT). When the system in SMT mode, the processor can be obtained from more than one instruction thread. SMT devoted to the concept of POWER5 architecture, it refers to a single process can not be used to implement all processor modules. POWER5 design cores in each chip to achieve a two-way SMT. As a result, each physical processor core that will be two virtual processors. SMT in the business environment is particularly useful, because in such an environment, the processing speed of a single transaction with the implementation of several services compared with the total not so important. With large or frequent changes in the work of collection (for example, database server and Web server), SMT should be able to increase the throughput of the workload. Very few floating-point intensive workloads to benefit from the SMT, which is most likely to reduce the performance factors. These will take up the workload of a large number of floating-point unit or memory bandwidth. The average per instruction clock cycles (CPI) and the cache hit rate is not lower workload may be a slight benefit from SMT. Under normal circumstances, SMT should be able to make the system performance increased by about 30%. You must be running on the system whether or not the key to the process to benefit from SMT. Can often be the key to the process to benefit from SMT; However, if you determine is not the case, you need to turn them off, because these processes are by default enabled.

Concept plan

I will try my best not to AIX kernel scheduler structures to spend too much time, but in introducing the process of optimizing the management or scheduling, you'll need for a better understanding of knowledge.

Each system has its own dedicated CPU run queue, the queue is based on thread priority value can sort the list of running threads. In addition, there is a run queue, called the global run queue. All new threads are running on the global queue. When ready to assign CPU thread, it will first check the overall operation of the queue, and then check the other run queue. In the CPU, when a thread to complete its time slice, it will run it back to the original run of the CPU queue. AIX This helps to maintain the relevance of its processor. (I will face in the post-processor to conduct a detailed discussion of the relevance.)

You can optimize some of the environmental variables to improve the performance of scheduler, but this discussion has gone beyond the scope of this article. CPU system shared by all threads, which is provided to each thread a certain amount of time slice to run. The default time slice is 10 milliseconds (one clock time unit). Schedo order to use the default values can be changed. Increase the time slice context switch can be reduced, so that we can improve the system throughput. Use vmstat or sar command to see the context switch. If the value of the context switch is very high, an increase of time-slicing can improve performance, but it must be through the implementation of a large number of analysis.

In the system model, CPU, there are two operation modes: kernel mode and user mode. In the user mode, the program dedicated to the process of the user data area has to read and write access. This model is the process of accumulation of most of its time where the CPU model. Another model is the kernel mode. Kernel-mode operation to some interrupt handling procedures, including procedures and core processes. When a process in the context of the implementation of this mode of operation of the code, it not only for the global kernel address space with read and write access, but also the core of the process data is also read and write access. Process address space for user data, core services must be used for a visit. When the user access system call, it is not in the kernel mode and user mode. You need to understand this concept, in order to try to explain, such as vmstat and sar command output.

Associated processor and processor bound

Relevance of the operating system processor is provided in the use of SMP hardware tools. In essence, the process of all the threads can be bound to the specified processor to run. AIX will be set up for each CPU on a run queue, in order to automatically try to promote the establishment of the relevance of the processor, which I have already discussed. Relevance through the use of process or thread to bind the abolition of the binding thread that can help you find it difficult or hang debugging the root cause of deadlock. For some applications, if they were always bound the thread of the CPU in a particular run, but also to improve its operating speed.

In a typical SMP system, all processors are identical and can run on their systems of arbitrary thread. In essence, any process or thread can be assigned to run on any processor, but bind in a specific processor or processor running, except thread. This can be done through the command bindprocessor. Let us look at an example (see list 1).

1. Bindprocessor order to use # bindprocessor -q
The available processors are:　0 1 2 3 # bindprocessor -q
The available processors are:　0 1 2 3

In this example, you can see there are four processors available for calls. These processors can be used is: 0,1,2,3.

The following command shows the bound to the processor CPU 3 (see list 2).

List 2. Found to bind to the CPU 3 of the processors # ps -emo THREAD | grep p3
root 401544 389152　　　　- A　　0　60　1 f10001001ece2fb8　 200001　pts/0
- grep p3 # ps -emo THREAD | grep p3
root 401544 389152　　　　- A　　0　60　1 f10001001ece2fb8　 200001　pts/0
- grep p3

You can also use the SMIT fast path smit bindproc to help the binding process. Another way is to bind the process, the use of AIX on bindprocessor API available in the internal implementation of the binding process. You should understand the function of these commands is very powerful. In the process bound to CPU if the CPU is in a busy state, while the other CPU is in idle state, the binding operation will actually lower the performance of the process.

PS - am glad to know you

Let us briefly discuss normally used to identify and deal with the order process.

To obtain a longer list of documents, use the following command, as shown in the list 3.

List 3. To obtain a long list of documents # ps -ef
UID　　PID　 PPID　 C　　STIME　　TTY　TIME CMD
root　　　1　　　0　 0　 Jan 08　　　-　0:05 /etc/init
root　82126 204974　 0　 Jan 08　　　-　0:00 /usr/sbin/snmpmibd
root　86210 106640　 0　 Jan 08　　　-　0:00 /usr/dt/bin/dtcm
root　90172 123038　 0　 Jan 08　　　-　0:35 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb
-T -force :0 -auth /var/dt/A:0-DjUjUa
root　98390　　　1　 0　 Jan 08　　　-　8:36 /usr/sbin/syncd 60
root 106640 131160　 0　 Jan 08　　　-　0:25 /usr/dt/bin/dtsession # ps -ef
UID　　PID　 PPID　 C　　STIME　　TTY　TIME CMD
root　　　1　　　0　 0　 Jan 08　　　-　0:05 /etc/init
root　82126 204974　 0　 Jan 08　　　-　0:00 /usr/sbin/snmpmibd
root　86210 106640　 0　 Jan 08　　　-　0:00 /usr/dt/bin/dtcm
root　90172 123038　 0　 Jan 08　　　-　0:35 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb
-T -force :0 -auth /var/dt/A:0-DjUjUa
root　98390　　　1　 0　 Jan 08　　　-　8:36 /usr/sbin/syncd 60
root 106640 131160　 0　 Jan 08　　　-　0:25 /usr/dt/bin/dtsession # ps -ef
UID　　PID　 PPID　 C　　STIME　　TTY　TIME CMD
root　　　1　　　0　 0　 Jan 08　　　-　0:05 /etc/init
root　82126 204974　 0　 Jan 08　　　-　0:00 /usr/sbin/snmpmibd
root　86210 106640　 0　 Jan 08　　　-　0:00 /usr/dt/bin/dtcm
root　90172 123038　 0　 Jan 08　　　-　0:35 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb
-T -force :0 -auth /var/dt/A:0-DjUjUa
root　98390　　　1　 0　 Jan 08　　　-　8:36 /usr/sbin/syncd 60
root 106640 131160　 0　 Jan 08　　　-　0:25 /usr/dt/bin/dtsession # ps -ef
UID　　PID　 PPID　 C　　STIME　　TTY　TIME CMD
root　　　1　　　0　 0　 Jan 08　　　-　0:05 /etc/init
root　82126 204974　 0　 Jan 08　　　-　0:00 /usr/sbin/snmpmibd
root　86210 106640　 0　 Jan 08　　　-　0:00 /usr/dt/bin/dtcm
root　90172 123038　 0　 Jan 08　　　-　0:35 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb
-T -force :0 -auth /var/dt/A:0-DjUjUa
root　98390　　　1　 0　 Jan 08　　　-　8:36 /usr/sbin/syncd 60
root 106640 131160　 0　 Jan 08　　　-　0:25 /usr/dt/bin/dtsession # ps -ef
UID　　PID　 PPID　 C　　STIME　　TTY　TIME CMD
root　　　1　　　0　 0　 Jan 08　　　-　0:05 /etc/init
root　82126 204974　 0　 Jan 08　　　-　0:00 /usr/sbin/snmpmibd
root　86210 106640　 0　 Jan 08　　　-　0:00 /usr/dt/bin/dtcm
root　90172 123038　 0　 Jan 08　　　-　0:35 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb
-T -force :0 -auth /var/dt/A:0-DjUjUa
root　98390　　　1　 0　 Jan 08　　　-　8:36 /usr/sbin/syncd 60
root 106640 131160　 0　 Jan 08　　　-　0:25 /usr/dt/bin/dtsession # ps -ef
UID　　PID　 PPID　 C　　STIME　　TTY　TIME CMD
root　　　1　　　0　 0　 Jan 08　　　-　0:05 /etc/init
root　82126 204974　 0　 Jan 08　　　-　0:00 /usr/sbin/snmpmibd
root　86210 106640　 0　 Jan 08　　　-　0:00 /usr/dt/bin/dtcm
root　90172 123038　 0　 Jan 08　　　-　0:35 /usr/lpp/X11/bin/X -D /usr/lib/X11//rgb
-T -force :0 -auth /var/dt/A:0-DjUjUa
root　98390　　　1　 0　 Jan 08　　　-　8:36 /usr/sbin/syncd 60
root 106640 131160　 0　 Jan 08　　　-　0:25 /usr/dt/bin/dtsession

To further identify the occupants of the CPU of the process of decomposition, see the list of 4.

List 4. Identification process # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23 # ps aux | more
USER　　　　PID %CPU %MEM　 SZ　RSS　　TTY STAT　　STIME　TIME COMMAND
root　　　 8196 12.9　0.0　384　384　　　- A　　　Jan 08 14695:30 wait
root　　　57372 12.8　0.0　384　384　　　- A　　　Jan 08 14542:51 wait
root　　　61470 12.2　0.0　384　384　　　- A　　　Jan 08 13884:38 wait
root　　　53274 12.0　0.0　384　384　　　- A　　　Jan 08 13711:38 wait
root　　 245938　0.0　0.0　828　856　　　- A　　　Jan 08 20:17 /usr/bin/xmwlm -
root　　　98390　0.0　0.0　508　516　　　- A　　　Jan 08　8:36 /usr/sbin/syncd
root　　　69666　0.0　0.0　960　960　　　- A　　　Jan 08　3:46 gil
root　　　　　0　0.0　0.0　384　384　　　- A　　　Jan 08　2:49 swapper
root　　　49176　0.0　0.0　448　448　　　- A　　　Jan 08　1:13 xmgc
root　　 241842　0.0　0.0 23

To learn more about the process if the priority value For more information, you need to use the-l mark. "NI" column shows the priority value (see the list of 5).

List of 5. Use-l to obtain the priority value signs # ps -elf
F　 S　 UID　 PID　 PPID　 c PRI　 NI ADDR　　SZ　 WCHAN　STIME　 TTY TIME　　　CMD
200003 A　 root　 1　　　0　　 0　　　60　20 14001400　660　 Jan 08　- 0:05　　/etc/init
240001 A　 root　82126 204974　0　　　60　20 3c22b510　1264　Jan 08　- 0:00 /usr/sbin/snmpmibd
240801 A　 root　86210 106640　0　　　60　20 584d2400　2156　Jan 08　- 0:00 /usr/dt/bin/dtcm
240001 A　 root　90172 123038　0　　　60　20 5136 f1000100224650e0　5136　Jan 08　- 0:35
/usr/lpp/X11/bin/X
-D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-DjUjUa
240001 A　 root　98390　　 1　 0　　　60　20 41a5400　 508 * Jan 08　- 8:36 /usr/sbin/syncd 60
240001 A　 root 106640 131160　0　　　60　20 3816a400　1880　Jan 08 - 0:25 /usr/dt/bin/dtsession
40001　A　 root 123038　　 1　 0　　　60　20 5c153400　 380　Jan 08-　0:00 /usr/dt/bin/dtlogin
-daemon # ps -elf
F　 S　 UID　 PID　 PPID　 C++ PRI　 NI ADDR　　SZ　 WCHAN　STIME　 TTY TIME　　　CMD
200003 A　 root　 1　　　0　　 0　　　60　20 14001400　660　 Jan 08　- 0:05　　/etc/init
240001 A　 root　82126 204974　0　　　60　20 3c22b510　1264　Jan 08　- 0:00 /usr/sbin/snmpmibd
240801 A　 root　86210 106640　0　　　60　20 584d2400　2156　Jan 08　- 0:00 /usr/dt/bin/dtcm
240001 A　 root　90172 123038　0　　　60　20 5136 f1000100224650e0　5136　Jan 08　- 0:35
/usr/lpp/X11/bin/X
-D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-DjUjUa
240001 A　 root　98390　　 1　 0　　　60　20 41a5400　 508 * Jan 08　- 8:36 /usr/sbin/syncd 60
240001 A　 root 106640 131160　0　　　60　20 3816a400　1880　Jan 08 - 0:25 /usr/dt/bin/dtsession
40001　A　 root 123038　　 1　 0　　　60　20 5c153400　 380　Jan 08-　0:00 /usr/dt/bin/dtlogin
-daemon # ps -elf
F　 S　 UID　 PID　 PPID　 C PRI　 NI ADDR　　SZ　 WCHAN　STIME　 TTY TIME　　　CMD
200003 A　 root　 1　　　0　　 0　　　60　20 14001400　660　 Jan 08　- 0:05　　/etc/init
240001 A　 root　82126 204974　0　　　60　20 3c22b510　1264　Jan 08　- 0:00 /usr/sbin/snmpmibd
240801 A　 root　86210 106640　0　　　60　20 584d2400　2156　Jan 08　- 0:00 /usr/dt/bin/dtcm
240001 A　 root　90172 123038　0　　　60　20 5136 f1000100224650e0　5136　Jan 08　- 0:35
/usr/lpp/X11/bin/X
-D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-DjUjUa
240001 A　 root　98390　　 1　 0　　　60　20 41a5400　 508 * Jan 08　- 8:36 /usr/sbin/syncd 60
240001 A　 root 106640 131160　0　　　60　20 3816a400　1880　Jan 08 - 0:25 /usr/dt/bin/dtsession
40001　A　 root 123038　　 1　 0　　　60　20 5c153400　 380　Jan 08-　0:00 /usr/dt/bin/dtlogin
-daemon # ps -elf
F　 S　 UID　 PID　 PPID　 C PRI　 NI ADDR　　SZ　 WCHAN　STIME　 TTY TIME　　　CMD
200003 A　 root　 1　　　0　　 0　　　60　20 14001400　660　 Jan 08　- 0:05　　/etc/init
240001 A　 root　82126 204974　0　　　60　20 3c22b510　1264　Jan 08　- 0:00 /usr/sbin/snmpmibd
240801 A　 root　86210 106640　0　　　60　20 584d2400　2156　Jan 08　- 0:00 /usr/dt/bin/dtcm
240001 A　 root　90172 123038　0　　　60　20 5136 f1000100224650e0　5136　Jan 08　- 0:35
/usr/lpp/X11/bin/X
-D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-DjUjUa
240001 A　 root　98390　　 1　 0　　　60　20 41a5400　 508 * Jan 08　- 8:36 /usr/sbin/syncd 60
240001 A　 root 106640 131160　0　　　60　20 3816a400　1880　Jan 08 - 0:25 /usr/dt/bin/dtsession
40001　A　 root 123038　　 1　 0　　　60　20 5c153400　 380　Jan 08-　0:00 /usr/dt/bin/dtlogin
-daemon # ps -elf
F　 S　 UID　 PID　 PPID　 C PRI　 NI ADDR　　SZ　 WCHAN　STIME　 TTY TIME　　　CMD
200003 A　 root　 1　　　0　　 0　　　60　20 14001400　660　 Jan 08　- 0:05　　/etc/init
240001 A　 root　82126 204974　0　　　60　20 3c22b510　1264　Jan 08　- 0:00 /usr/sbin/snmpmibd
240801 A　 root　86210 106640　0　　　60　20 584d2400　2156　Jan 08　- 0:00 /usr/dt/bin/dtcm
240001 A　 root　90172 123038　0　　　60　20 5136 f1000100224650e0　5136　Jan 08　- 0:35
/usr/lpp/X11/bin/X
-D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-DjUjUa
240001 A　 root　98390　　 1　 0　　　60　20 41a5400　 508 * Jan 08　- 8:36 /usr/sbin/syncd 60
240001 A　 root 106640 131160　0　　　60　20 3816a400　1880　Jan 08 - 0:25 /usr/dt/bin/dtsession
40001　A　 root 123038　　 1　 0　　　60　20 5c153400　 380　Jan 08-　0:00 /usr/dt/bin/dtlogin
-daemon # ps -elf
F　 S　 UID　 PID　 PPID　 C PRI　 NI ADDR　　SZ　 WCHAN　STIME　 TTY TIME　　　CMD
200003 A　 root　 1　　　0　　 0　　　60　20 14001400　660　 Jan 08　- 0:05　　/etc/init
240001 A　 root　82126 204974　0　　　60　20 3c22b510　1264　Jan 08　- 0:00 /usr/sbin/snmpmibd
240801 A　 root　86210 106640　0　　　60　20 584d2400　2156　Jan 08　- 0:00 /usr/dt/bin/dtcm
240001 A　 root　90172 123038　0　　　60　20 5136 f1000100224650e0　5136　Jan 08　- 0:35
/usr/lpp/X11/bin/X
-D /usr/lib/X11//rgb -T -force :0 -auth /var/dt/A:0-DjUjUa
240001 A　 root　98390　　 1　 0　　　60　20 41a5400　 508 * Jan 08　- 8:36 /usr/sbin/syncd 60
240001 A　 root 106640 131160　0　　　60　20 3816a400　1880　Jan 08 - 0:25 /usr/dt/bin/dtsession
40001　A　 root 123038　　 1　 0　　　60　20 5c153400　 380　Jan 08-　0:00 /usr/dt/bin/dtlogin
-daemon

Ordered list of 6 provided the first three of the process of being implemented, and gives priority to their value.

List 6. Access to the former three processes are running # ps -elf | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 15
40401 A　 nobody 323762 127128　 0　60 20 602dc400　 660 f1000600002daa08　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 319662 127128　 0　60 20 6c35f400　1336　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 307358 127128　 0　60 20 3834a400　1340　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
240001 A　 daemon 254084 204974　 0　60 20 58272400　1364　　　　　　Jan 08　　　-　0:00
/usr/sbin/rpc.statd -d 0 -t 50 # ps -elf | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 15
40401 A　 nobody 323762 127128　 0　60 20 602dc400　 660 f1000600002daa08　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 319662 127128　 0　60 20 6c35f400　1336　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 307358 127128　 0　60 20 3834a400　1340　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
240001 A　 daemon 254084 204974　 0　60 20 58272400　1364　　　　　　Jan 08　　　-　0:00
/usr/sbin/rpc.statd -d 0 -t 50 # ps -elf | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 15
40401 A　 nobody 323762 127128　 0　60 20 602dc400　 660 f1000600002daa08　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 319662 127128　 0　60 20 6c35f400　1336　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 307358 127128　 0　60 20 3834a400　1340　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
240001 A　 daemon 254084 204974　 0　60 20 58272400　1364　　　　　　Jan 08　　　-　0:00
/usr/sbin/rpc.statd -d 0 -t 50 # ps -elf | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 15
40401 A　 nobody 323762 127128　 0　60 20 602dc400　 660 f1000600002daa08　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 319662 127128　 0　60 20 6c35f400　1336　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 307358 127128　 0　60 20 3834a400　1340　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
240001 A　 daemon 254084 204974　 0　60 20 58272400　1364　　　　　　Jan 08　　　-　0:00
/usr/sbin/rpc.statd -d 0 -t 50 # ps -elf | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 15
40401 A　 nobody 323762 127128　 0　60 20 602dc400　 660 f1000600002daa08　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 319662 127128　 0　60 20 6c35f400　1336　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
40001 A　 nobody 307358 127128　 0　60 20 3834a400　1340　　　　*　 Jan 08　　　-　0:00
/usr/HTTPServer/bin/httpd -d /usr/HTTPServer -k restart
240001 A　 daemon 254084 204974　 0　60 20 58272400　1364　　　　　　Jan 08　　　-　0:00
/usr/sbin/rpc.statd -d 0 -t 50

Now, you know what the termination of the system process (you can also use topas or nmon), following the how to deal with that? If there is a command allows you to plan for the core set of priorities to deal with, is not on a perfect? Of course there is such a command, not only that, there is another command allows you to run the process has once again set the priority. These two commands are nice and renice. In AIX, the user operation of the basic priority of 40, the default priority value of 20. The two figures together constitute a default priority of 60. The vast majority of operations are the use of this value. The default priority number the higher the priority of operations lower. If you want to start a lower-priority operation, you can try to order a list of 7.

List of 7. To start with lower-priority operation # nice -n 10 thisjob

7 in the list of default values in the order added 20 on 10, to create a new priority value of 30 and priority 70.

Running list of 8 in the 1683 order resulted in the process of priority value to 30.

A list of 8. Lead to the process priority value 1683 into a 30 # renice -n 10 -p 1683

utility procmon

Although the basic operating system, AIX comes with a lot of performance tools, but in the recent introduction of performance monitoring tools is the best possible procmon (introduced in AIX 5.3). This utility will show the process of a dynamic list, this list has been sorted, to be able to provide all the information in these processes. It allows the implementation of nice, renice and kill commands, such as basic management. Performance Workbench tool procmon platform, the tool based on Eclipse, and has a small graphical user interface to display system activity. To start procmon, please start perfwb, which will start with procmon plug Eclipse (see list 9). During this process, you need to use bos.perf.gtools.perfwb documentation.

The list 9. Perfwb start # /usr/bin/perfwb

By default, procmon tool will show the following:

Process has to run the length of time

Process is the use of CPU resources

The process of being punished by the system

Process is the amount of memory used

The process of implementation of the I / O number

Process priority and a priority value

Staff to create a specific process

In addition, it has the following options for the implementation of:

procfiles

proctree

procsig

procstack

procrun

procmap

procflags

proccred

procldd

Process procmon table is the main component, it shows the system is running on all the process, you can configure on the basis of the user to sort and filter them. By default, the process of the process table are 20, but the main menu in the Table Properties panel, the user can easily change the process of a few. About this important tool for more information, see the reference.

WLM

WLM is a complex tool that can be used to monitor performance, collect data, and managed as a separate account on the system load. Environment in the District, the tool also can be used (with the DLPAR) resources provide the tools. To the system administrator, this can be an effective process to monitor and control the use of resources.

AIX's WLM provides a series of useful tools, they not only contribute to the collection of performance statistics can also provide you with a mechanism to control the resource allocation process. This feature is available for running multiple applications, databases and transaction processing system for large-scale systems, in which all the work load has been merged into a single large-scale system ( "vertical" integration server). It provides a division between the operating system resources on the system partition without flexibility. The use of WLM, you can not only prevent the operation of different types of interference can also be based on the requirements of different user groups to allocate resources. Many people will load WLM and Zoning Manager (PLM) with confusion. Indeed, PLM is a resource manager, which is based on the defined strategy and resource utilization in the IBM System p ™ environment and mobile distribution of resources, including the High power Virtualization. PLM technology to re-use micro-adjustment of resources, can be used for the management of memory, dedicated processor partitions and shared processor partition. This, in turn, provided for the POWER Hypervisor to increase the flexibility of micro-flexibility. Unfortunately, PLM did not take into account the operation of any district in the importance of workload, it can not be based on the types of changes to the workload of re-adjustment of priority.

Concluding remarks

On the UNIX system administrator, the process management and a far cry from the work of the most exciting. Although it tedious, it is annoying, but it is necessary for system management. You still need to answer questions about the process of management of a variety of issues: how to speed up the processing speed of the process? Why is this process taking so long to complete? You need to identify process problems, and to take all measures to improve the operating efficiency of the process. In addition, you need to know to complete the work of the best tools, you can only run the ps command and use the renice, etc. You can also use the new procmon utility performance, and even plans to introduce the process of enterprise-class practical tools (such as WLM), in order to more effectively manage the process of all systems. With the introduction of any new element, please try to plan for the core process and the concept of a number of other studies. At the beginning of work, to really understand the work involved in the long run will help you have more.

 

end