`
netcome
  • 浏览: 482100 次
  • 性别: Icon_minigender_1
  • 来自: 北京
社区版块
存档分类
最新评论

Understanding memory usage on Linux

阅读更多

This entry is for those people who have ever wondered, "Why the hell is a simple KDE text editor taking up 25 megabytes of memory?" Many people are led to believe that many Linux applications, especially KDE or Gnome programs, are "bloated" based solely upon what tools like ps report. While this may or may not be true, depending on the program, it is not generally true -- many programs are much more memory efficient than they seem.

What ps reports
The ps tool can output various pieces of information about a process, such as its process id, current running state, and resource utilization. Two of the possible outputs are VSZ and RSS, which stand for "virtual set size" and "resident set size", which are commonly used by geeks around the world to see how much memory processes are taking up.

For example, here is the output of ps aux for KEdit on my computer:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
dbunker   3468  0.0  2.7  25400 14452 ?        S    20:19   0:00 kdeinit: kedit

According to ps, KEdit has a virtual size of about 25 megabytes and a resident size of about 14 megabytes (both numbers above are reported in kilobytes). It seems that most people like to randomly choose to accept one number or the other as representing the real memory usage of a process. I'm not going to explain the difference between VSZ and RSS right now but, needless to say, this is the wrong approach; neither number is an accurate picture of what the memory cost of running KEdit is.

Why ps is "wrong"
Depending on how you look at it, ps is not reporting the real memory usage of processes. What it is really doing is showing how much real memory each process would take up if it were the only process running. Of course, a typical Linux machine has several dozen processes running at any given time, which means that the VSZ and RSS numbers reported by ps are almost definitely "wrong". In order to understand why, it is necessary to learn how Linux handles shared libraries in programs.

Most major programs on Linux use shared libraries to facilitate certain functionality. For example, a KDE text editing program will use several KDE shared libraries (to allow for interaction with other KDE components), several X libraries (to allow it to display images and copy and pasting), and several general system libraries (to allow it to perform basic operations). Many of these shared libraries, especially commonly used ones like libc, are used by many of the programs running on a Linux system. Due to this sharing, Linux is able to use a great trick: it will load a single copy of the shared libraries into memory and use that one copy for every program that references it.

For better or worse, many tools don't care very much about this very common trick; they simply report how much memory a process uses, regardless of whether that memory is shared with other processes as well. Two programs could therefore use a large shared library and yet have its size count towards both of their memory usage totals; the library is being double-counted, which can be very misleading if you don't know what is going on.

Unfortunately, a perfect representation of process memory usage isn't easy to obtain. Not only do you need to understand how the system really works, but you need to decide how you want to deal with some hard questions. Should a shared library that is only needed for one process be counted in that process's memory usage? If a shared library is used my multiple processes, should its memory usage be evenly distributed among the different processes, or just ignored? There isn't a hard and fast rule here; you might have different answers depending on the situation you're facing. It's easy to see why ps doesn't try harder to report "correct" memory usage totals, given the ambiguity.

Seeing a process's memory map
Enough talk; let's see what the situation is with that "huge" KEdit process. To see what KEdit's memory looks like, we'll use the pmap program (with the -d flag):
Address   Kbytes Mode  Offset           Device    Mapping
08048000      40 r-x-- 0000000000000000 0fe:00000 kdeinit
08052000       4 rw--- 0000000000009000 0fe:00000 kdeinit
08053000    1164 rw--- 0000000008053000 000:00000   [ anon ]
40000000      84 r-x-- 0000000000000000 0fe:00000 ld-2.3.5.so
40015000       8 rw--- 0000000000014000 0fe:00000 ld-2.3.5.so
40017000       4 rw--- 0000000040017000 000:00000   [ anon ]
40018000       4 r-x-- 0000000000000000 0fe:00000 kedit.so
40019000       4 rw--- 0000000000000000 0fe:00000 kedit.so
40027000     252 r-x-- 0000000000000000 0fe:00000 libkparts.so.2.1.0
40066000      20 rw--- 000000000003e000 0fe:00000 libkparts.so.2.1.0
4006b000    3108 r-x-- 0000000000000000 0fe:00000 libkio.so.4.2.0
40374000     116 rw--- 0000000000309000 0fe:00000 libkio.so.4.2.0
40391000       8 rw--- 0000000040391000 000:00000   [ anon ]
40393000    2644 r-x-- 0000000000000000 0fe:00000 libkdeui.so.4.2.0
40628000     164 rw--- 0000000000295000 0fe:00000 libkdeui.so.4.2.0
40651000       4 rw--- 0000000040651000 000:00000   [ anon ]
40652000     100 r-x-- 0000000000000000 0fe:00000 libkdesu.so.4.2.0
4066b000       4 rw--- 0000000000019000 0fe:00000 libkdesu.so.4.2.0
4066c000      68 r-x-- 0000000000000000 0fe:00000 libkwalletclient.so.1.0.0
4067d000       4 rw--- 0000000000011000 0fe:00000 libkwalletclient.so.1.0.0
4067e000       4 rw--- 000000004067e000 000:00000   [ anon ]
4067f000    2148 r-x-- 0000000000000000 0fe:00000 libkdecore.so.4.2.0
40898000      64 rw--- 0000000000219000 0fe:00000 libkdecore.so.4.2.0
408a8000       8 rw--- 00000000408a8000 000:00000   [ anon ]
... (trimmed) ...
mapped: 25404K    writeable/private: 2432K    shared: 0K


I cut out a lot of the output; the rest is similar to what is shown. Even without the complete output, we can see some very interesting things. One important thing to note about the output is that each shared library is listed twice; once for its code segment and once for its data segment. The code segments have a mode of "r-x--", while the data is set to "rw---". The Kbytes, Mode, and Mapping columns are the only ones we will care about, as the rest are unimportant to the discussion.

If you go through the output, you will find that the lines with the largest Kbytes number are usually the code segments of the included shared libraries (the ones that start with "lib" are the shared libraries). What is great about that is that they are the ones that can be shared between processes. If you factor out all of the parts that are shared between processes, you end up with the "writeable/private" total, which is shown at the bottom of the output. This is what can be considered the incremental cost of this process, factoring out the shared libraries. Therefore, the cost to run this instance of KEdit (assuming that all of the shared libraries were already loaded) is around 2 megabytes. That is quite a different story from the 14 or 25 megabytes that ps reported.

What does it all mean?
The moral of this story is that process memory usage on Linux is a complex matter; you can't just run ps and know what is going on. This is especially true when you deal with programs that create a lot of identical children processes, like Apache. ps might report that each Apache process uses 10 megabytes of memory, when the reality might be that the marginal cost of each Apache process is 1 megabyte of memory. This information becomes critial when tuning Apache's MaxClients setting, which determines how many simultaneous requests your server can handle (although see one of my past postings for another way of increasing Apache's performance).

It also shows that it pays to stick with one desktop's software as much as possible. If you run KDE for your desktop, but mostly use Gnome applications, then you are paying a large price for a lot of redundant (but different) shared libraries. By sticking to just KDE or just Gnome apps as much as possible, you reduce your overall memory usage due to the reduced marginal memory cost of running new KDE or Gnome applications, which allows Linux to use more memory for other interesting things (like the file cache, which speeds up file accesses immensely).

 

http://virtualthreads.blogspot.com/2006/02/understanding-memory-usage-on-linux.html

分享到:
评论

相关推荐

    understanding the linux virtual memory manager

    《深入理解Linux虚拟内存管理》 在Linux操作系统中,虚拟内存管理是系统核心的重要组成部分,它使得进程可以独立地使用内存,而无需关心物理内存的实际分布情况。这一机制的实现,使得Linux系统能够高效地利用有限...

    Understanding The Linux Virtual Memory Manager.rar

    Linux虚拟内存管理系统(Virtual Memory Manager,VMM)是Linux内核的核心组件之一,它负责为进程提供独立、私有的地址空间,并通过内存映射、交换机制等手段优化物理内存的使用。 Linux虚拟内存的主要目标是提供一...

    Understanding the Linux Kernel pdf

    4. 《UnderStanding The Linux Kernel 3rd Edition V413HAV.pdf》:这正是标题中提到的《理解Linux内核》第三版,书中详尽介绍了Linux 2.6.x版本内核的设计和实现,包括调度程序、内存管理、文件系统、网络协议栈等...

    Understanding_the_Linux_Virtual_Memory_Manager

    ### 理解 Linux 虚拟内存管理器 #### 引言 本文旨在深入剖析 Linux 内核中的虚拟内存管理机制。对于操作系统而言,内存管理是其核心功能之一,尤其是在现代多任务处理环境中,有效地管理和分配内存资源至关重要。...

    Understanding Memory

    2. 相关技术书籍,如《Understanding the Linux Kernel》 3. 在线课程和教程 通过以上知识点的详细解释,我们对Linux下的内存管理有了较为全面的理解。虚拟内存机制不仅提供了内存隔离和保护,还极大地简化了程序...

    Understanding the Linux Virtual Memory Manager

    Linux虚拟存储管理器(Virtual Memory Manager,VMM)的作用是高效地管理内存,使得每个运行的程序都认为它们独占了整个系统的内存空间。然而,实际上,物理内存的数量是有限的,Linux通过虚拟内存技术来克服这一...

    Understanding Linux Virtual Memory

    Linux虚拟内存是Linux操作系统中非常重要的一个概念,它是现代操作系统内存管理技术的核心部分。理解Linux虚拟内存的管理,对掌握Linux系统的工作原理,进行系统优化和故障排查有重要作用。 Linux虚拟内存管理主要...

    Understanding Unix/Linux Programming

    《理解Unix/Linux编程》这本书是Unix/Linux编程领域的经典之作,旨在帮助读者深入理解这两个操作系统的核心原理和编程技术。书中的内容不仅涵盖了基础的系统调用、文件操作,还包括进程管理、网络编程等高级主题,是...

    了解Linux虚拟内存管理器Understanding the Linux Virtual Memory Manager

    详细描述Linux虚拟内存,提供理论基础和逐行源代码注释。 它系统地涵盖了从物理内存描述到内存不足管理的所有内容。

    Direct Memory Access in Linux.doc

    The original poster is working on an embedded Linux project where they need to access physical memory directly for implementing a high-performance logging solution. The goal is to allocate a specific ...

    Understanding Memory Resource Management in VMware® ESX™ Server

    更详细的资料可参阅“Memory Resource Management in VMware ESX Server”。 ESX利用高级资源管理策略计算每个虚拟机的目标内存分配,该分配基于当前系统负载以及虚拟机的设置参数(如份额、预留和限制)。计算出的...

    Understanding the Linux Kernel 3rd Edition

    该压缩包中的文件"OReilly.Understanding.the.Linux.Kernel.3rd.Edition.Nov.2005.HAPPY.NEW.YEAR.chm"是这本书的电子版,采用CHM(Compiled HTML Help)格式,便于在电脑上离线阅读。CHM文件是一种包含HTML页面和...

    Understanding_the_Linux_Kernel_3rd 深入理解LINUX内核. Linux内核注释

    Understanding_the_Linux_Kernel_3rd 深入理解LINUX内核. Linux内核注释.rar Understanding_the_Linux_Kernel_3rd 深入理解LINUX内核. Linux内核注释.rar Understanding_the_Linux_Kernel_3rd 深入理解LINUX内核. ...

    Understanding_The_Linux_Virtual_Memory_Manager

    available online, see Understand The Linux Virtual Memory Manager, online This book is specifically dedicated to the virtual memory manager of the Linux kernel, and so goes into deep details about ...

    Understanding.the.Linux.Kernel.rar

    Understanding.the.Linux.Kernel.rar Understanding.the.Linux.Kernel.rar Understanding.the.Linux.Kernel.rar

Global site tag (gtag.js) - Google Analytics