[os] Kernel是如何管理你的内存 How The Kernel Manages Your Memory

转自:Melody_lu123 CSDN 博客 ,很赞的技术文章.

原文作者:Gustavo Duarte


How The Kernel Manages Your Memory


After examining the virtual address layout of a process, we turn to the kernel and its mechanisms for managing user memory. Here is gonzo again:


Linux kernel mm_struct

Linux processes are implemented in the kernel as instances of task_struct, the process descriptor. The mm field in task_struct points to the memory descriptormm_struct, which is an executive summary of a program’s memory. It stores the start and end of memory segments as shown above, the number of physical memory pages used by the process (rss stands for Resident Set Size), the amount of virtual address space used, and other tidbits. Within the memory descriptor we also find the two work horses for managing program memory: the set of virtual memory areas and thepage tables. Gonzo’s memory areas are shown below:

Linux的进程在kernel中是由task_struct来实现的。其中有一个mm域指向这个进程所使用的内存的描述符,mm_struct。它包含了如上所示的各个段的开始和结束的地址(使用的是unsigned long类型来表示),被这个进程所使用的物理内存页的个数,各种被使用到的虚拟地址空间(total_vm, shared_vm,...)和其它一些内容。从这个结构体中我们能看到两块主要的用来管理程序内存的成员:虚拟内存空间(vm_area_struct)和页表(pgd,这是一个平台相关的结构)。下图是Gonzo的虚拟内存的布局。

Kernel memory descriptor and memory areas

Each virtual memory area (VMA) is a contiguous range of virtual addresses; these areas never overlap. An instance of vm_area_struct fully describes a memory area, including its start and end addresses, flags to determine access rights and behaviors, and the vm_file field to specify which file is being mapped by the area, if any. A VMA that does not map a file is anonymous. Each memory segment above (e.g., heap, stack) corresponds to a single VMA, with the exception of the memory mapping segment. This is not a requirement, though it is usual in x86 machines. VMAs do not care which segment they are in.

每个虚拟内存区域都是一个连续的虚拟地址空间;这些区域是不会重叠的。一个vm_area_struct表示一个这样的内存区域,包括该区域的起始和结束地址,flags描述了它的访问权限和行为,vm_file表示哪一个文件被映射到这块区域。一个VMA没有映射一个文件被称为是匿名的。上面的每个内存段(堆,栈...)都对应着一个VMA,唯一的例外是memory mapping segment,它会有几个VMA来描述。这虽然不是必须的,但是在x86下一般都是这样。任何一个VMA是不关心它是属于哪个段的。

A program’s VMAs are stored in its memory descriptor both as a linked list in the mmap field, ordered by starting virtual address, and as a red-black tree rooted at the mm_rb field. The red-black tree allows the kernel to search quickly for the memory area covering a given virtual address. When you read file /proc/pid_of_process/maps, the kernel is simply going through the linked list of VMAs for the process and printing each one.


In Windows, the EPROCESS block is roughly a mix of task_struct and mm_struct. The Windows analog to a VMA is the Virtual Address Descriptor, or VAD; they are stored in an AVL tree. You know what the funniest thing about Windows and Linux is? It’s the little differences.

在windows中,使用EPROCESS结构来描述linux中的task_struct和mm_struct。windows中与VMA相似的概念是Virtual Address Descriptor,或则叫做VAD;它们被AVL树所管理。你知道关于windows和linux最搞笑的事情吗?那就是它们在虚拟内存方面只有很小的区别。

The 4GB virtual address space is divided into pages. x86 processors in 32-bit mode support page sizes of 4KB, 2MB, and 4MB. Both Linux and Windows map the user portion of the virtual address space using 4KB pages. Bytes 0-4095 fall in page 0, bytes 4096-8191 fall in page 1, and so on. The size of a VMA must be a multiple of page size. Here’s 3GB of user space in 4KB pages:


4KB Pages Virtual User Space

The processor consults page tables to translate a virtual address into a physical memory address. Each process has its own set of page tables; whenever a process switch occurs, page tables for user space are switched as well. Linux stores a pointer to a process’ page tables in the pgd field of the memory descriptor. To each virtual page there corresponds one page table entry (PTE) in the page tables, which in regular x86 paging is a simple 4-byte record shown below:


x86 Page Table Entry (PTE) for 4KB page

Linux has functions to read and set each flag in a PTE. Bit P tells the processor whether the virtual page is present in physical memory. If clear (equal to 0), accessing the page triggers a page fault. Keep in mind that when this bit is zero, the kernel can do whatever it pleases with the remaining fields. The R/W flag stands for read/write; if clear, the page is read-only. Flag U/S stands for user/supervisor; if clear, then the page can only be accessed by the kernel. These flags are used to implement the read-only memory and protected kernel space we saw before.


Bits D and A are for dirty and accessed. A dirty page has had a write, while an accessed page has had a write or read. Both flags are sticky: the processor only sets them, they must be cleared by the kernel. Finally, the PTE stores the starting physical address that corresponds to this page, aligned to 4KB. This naive-looking field is the source of some pain, for it limits addressable physical memory to 4 GB. The other PTE fields are for another day, as is Physical Address Extension.


A virtual page is the unit of memory protection because all of its bytes share the U/S and R/W flags. However, the same physical memory could be mapped by different pages, possibly with different protection flags. Notice that execute permissions are nowhere to be seen in the PTE. This is why classic x86 paging allows code on the stack to be executed, making it easier to exploit stack buffer overflows (it’s still possible to exploit non-executable stacks using return-to-libc and other techniques). This lack of a PTE no-execute flag illustrates a broader fact: permission flags in a VMA may or may not translate cleanly into hardware protection. The kernel does what it can, but ultimately the architecture limits what is possible.


Virtual memory doesn’t store anything, it simply maps a program’s address space onto the underlying physical memory, which is accessed by the processor as a large block called the physical address space. While memory operations on the bus are somewhat involved, we can ignore that here and assume that physical addresses range from zero to the top of available memory in one-byte increments. This physical address space is broken down by the kernel into page frames. The processor doesn’t know or care about frames, yet they are crucial to the kernel because the page frame is the unit of physical memory management. Both Linux and Windows use 4KB page frames in 32-bit mode; here is an example of a machine with 2GB of RAM:


Physical Address Space

In Linux each page frame is tracked by a descriptor and several flags. Together these descriptors track the entire physical memory in the computer; the precise state of each page frame is always known. Physical memory is managed with the buddy memory allocation technique, hence a page frame is free if it’s available for allocation via the buddy system. An allocated page frame might be anonymous, holding program data, or it might be in the page cache, holding data stored in a file or block device. There are other exotic page frame uses, but leave them alone for now. Windows has an analogous Page Frame Number (PFN) database to track physical memory.

linux中每个页帧是通过一个描述符(即page struct)和一些标志来追踪。而通过把这些描述符结合起来来追踪计算机内的所有物理内存。每个页帧的具体状态是完全可知的。物理内存是通过伙伴内存分配系统来管理的,因此如果一个页帧对于伙伴系统来说是可用的那么这个页帧就是空闲的。一个被分配出来的页帧可能是匿名的,可以用来保存程序数据或者可能是一个页cache,或者保存一个文件中的数据或是一个块设备。还有一些其它的特别的页帧的使用,但是我们先不考虑它们。Windows有一个类似的Page Frame Number(PFN)的数据库来帮助跟踪物理内存。

Let’s put together virtual memory areas, page table entries and page frames to understand how this all works. Below is an example of a user heap:


Physical Address Space

Blue rectangles represent pages in the VMA range, while arrows represent page table entries mapping pages onto page frames. Some virtual pages lack arrows; this means their corresponding PTEs have the Present flag clear. This could be because the pages have never been touched or because their contents have been swapped out. In either case access to these pages will lead to page faults, even though they are within the VMA. It may seem strange for the VMA and the page tables to disagree, yet this often happens.


A VMA is like a contract between your program and the kernel. You ask for something to be done (memory allocated, a file mapped, etc.), the kernel says “sure”, and it creates or updates the appropriate VMA. But it does not actually honor the request right away, it waits until a page fault happens to do real work. The kernel is a lazy, deceitful sack of scum; this is the fundamental principle of virtual memory. It applies in most situations, some familiar and some surprising, but the rule is that VMAs record what has been agreed upon, while PTEs reflect what has actually been done by the lazy kernel. These two data structures together manage a program’s memory; both play a role in resolving page faults, freeing memory, swapping memory out, and so on. Let’s take the simple case of memory allocation:


Example of demand paging and memory allocation

When the program asks for more memory via the brk() system call, the kernel simply updates the heap VMA and calls it good. No page frames are actually allocated at this point and the new pages are not present in physical memory. Once the program tries to access the pages, the processor page faults and do_page_fault() is called. It searches for the VMA covering the faulted virtual address using find_vma(). If found, the permissions on the VMA are also checked against the attempted access (read or write). If there’s no suitable VMA, no contract covers the attempted memory access and the process is punished by Segmentation Fault.

当一个程序通过brk系统调用要求更多的内存时,内核只是简单的更新了它的堆所在的VMA。没有页帧被真正的分配出来,并且新的页是没有在物理内存上建立出来。一旦程序尝试访问这些页,则处理器出发一个页错误,即执行do_page_fault()函数。它通过find_vma()来找到导致页错误的VMA中的虚拟地址。如果找到了,且各种VMA中的权限检查没有通过,则表示没有合适的VMA,即没有建立好适合的契约,那么这次的内存访问就是非法的,会导致内核发出Segmentation Fault。

When a VMA is found the kernel must handle the fault by looking at the PTE contents and the type of VMA. In our case, the PTE shows the page is not present. In fact, our PTE is completely blank (all zeros), which in Linux means the virtual page has never been mapped. Since this is an anonymous VMA, we have a purely RAM affair that must be handled by do_anonymous_page(), which allocates a page frame and makes a PTE to map the faulted virtual page onto the freshly allocated frame.

当VMA被找到,并且各种权限检查通过,则PTE会表示该页不在现有页帧当中。在我们的例子中,我们的PTE是完全空的,这意味着linux kernel认为虚拟页还没有被映射。因为这是一个匿名的VMA,我们就必须使用do_anonymous_page()处理这种情况,它会把之前PTE的错误的虚拟页映射到新分配出来的页帧上。

Things could have been different. The PTE for a swapped out page, for example, has 0 in the Present flag but is not blank. Instead, it stores the swap location holding the page contents, which must be read from disk and loaded into a page frame by do_swap_page() in what is called a major fault.

当然,事情也有可能有所区别。比如PTE中是一个换出的页,此时Present标示是0但是该PTE却不是空的。相反,它存储的交换区的地址里面保存的是页内容,它必须由do_swap_page()从硬盘里读取到页帧中。这叫做major fault

This concludes the first half of our tour through the kernel’s user memory management. In the next post, we’ll throw files into the mix to build a complete picture of memory fundamentals, including consequences for performance.


posted @ 2012-04-13 16:25  zsounder  阅读(1226)  评论(0编辑  收藏  举报