Kernel Memory Chris Gill David Ferry Brian Kocoloski
Kernel Memory Chris Gill, David Ferry, Brian Kocoloski CSE 522 S - Advanced Operating Systems Washington University in St. Louis, MO 63130 1
Kernel vs User Virtual Memory • When page tables are filled in on-demand like this, this is referred to as demand paging • By default, Linux uses demand paging for all user processes – Keeps memory allocation (virtual, i. e. , malloc()) fast – Doesn’t allocate physical memory until process tries to access it • However, kernel virtual memory is always prepaged – When the system boots, all physical memory is mapped into the kernel virtual address space CSE 522 S – Advanced Operating Systems 2
Kernel vs User Virtual Memory Kernel Virtual Memory User Virtual Memory Physical Memory Mappings created at runtime via memory allocation (e. g. , mmap()) and page fault handling Mappings created at system boot time CSE 522 S – Advanced Operating Systems 3
Traditional Memory Mapping Strategy in Linux (on x 86_64) Virtual Memory Kernel address space Mappings created at system boot time Physical Memory CSE 522 S – Advanced Operating Systems 4
Traditional Memory Mapping Strategy in Linux (on x 86_64) Virtual Memory User address space Kernel address space Mappings created at system boot time Mappings created at runtime via memory allocation (e. g. , mmap()) and page fault handling Physical Memory CSE 522 S – Advanced Operating Systems 5
Traditional Memory Mapping Strategy in Linux (on x 86_64) Virtual Memory User address space Kernel address space • Green VA are free • Blue VA are allocated • But, both area always mapped in the kernel’s page tables Physical Memory CSE 522 S – Advanced Operating Systems 6
Physical to Virtual Translation • Given a physical address, how do I calculate the kernel virtual address to access it? – __va(unsigned long pa); – On arm 7 (R Pi 3): ((unsigned long)((x) - PHYS_OFFSET + PAGE_OFFSET)) – PHYS_OFFSET: first address of physical memory – PAGE_OFFSET: (On our 32 -bit R Pis) 0 x 80000000 • The point is, reverse PA->VA translation can be done in O(1) time CSE 422 S – Operating Systems Organization 7
Dynamic Management of Kernel Memory Page level management – Use alloc_pages() and free_pages() etc. Object level management – Use kmalloc() and kfree() etc. (virtually and physically contiguous) – Use vmalloc() and vfree() etc. (only virtually contiguous ) Slab allocation – Use kmem_cache_create() and then kmem_cache_alloc() and kmem_cache_free() etc. for each distinct type of memory object CSE 522 S – Advanced Operating Systems 8
Page Level Management (Pre-)allocating larger chunks of memory – E. g. , 2 n contiguous pages: alloc_pages(GFP_KERNEL, n) – Each page is fixed size (usually 4 Kb or 8 Kb) with enough room for hundreds of individual objects Can request a page with filled-in values – Use get_zeroed_page() to hide previous contents Must free pages once finished using them – Use free_page() if a single page was allocated – Use free_pages() if multiple were allocated at once CSE 522 S – Advanced Operating Systems 9
Object Level Management Size of each allocation is in bytes – I. e. , the size of the object being allocated – Rather than the number of fixed-sized pages Kernel version of malloc: kmalloc() and kfree() – – Action modifiers (can/cannot sleep, etc. ) Zone modifiers (DMA, DMA 32, HIGH, NORMAL) Type flags: combine action and zone modifiers Allocated memory is physically contiguous Virtually contiguous memory – Though not necessarily physically contiguous – Use vmalloc() and vfree() etc. CSE 522 S – Advanced Operating Systems 10
Traditional Memory Mapping Strategy in Linux (on x 86_64) Virtual Memory Assume green VA are free Assume blue VA are allocated Kernel address space Physical Memory Calls to kmalloc() effectively search through green VA space to find free virtual address space CSE 522 S – Advanced Operating Systems 11
Traditional Memory Mapping Strategy in Linux (on x 86_64) Virtual Memory Assume blue VA are allocated Calls to vmalloc() effectively allocate new, previously unused virtual address space, and map it to possibly non-contiguous memory Kernel address space Physical Memory New vmalloc()’d memory CSE 522 S – Advanced Operating Systems 12
Kernel Memory Trade-Offs Page level management – Coarse granularity, low performance overhead Object level management – Finer granularity, higher performance overhead – By default, prefer the kmalloc() version over the vmalloc() version, for performance Slab allocation – Optimizes performance if allocating and deallocating large numbers of the same kind of object CSE 522 S – Advanced Operating Systems 13
Other Kernel Memory Options Static allocation on the stack – Can be risky, as stacks are limited to 1 or 2 pages – In general, minimize stack usage by design – By default, use the kernel allocation techniques previously discussed instead of stack allocation Per-CPU allocation – – Think of this as “core-specific storage” Can improve cache performance, reduce contention Must disable kernel preemption (per-CPU interface does) Must “alloc” and “put” per-CPU data (like alloc/free) CSE 522 S – Advanced Operating Systems 14
Today’s Studio(s) 1. Cross-compile and deploy a simple “Hello World” Linux kernel module, slightly modify it to get a feel for how it’s done 2. Explore kernel memory management by requesting, modifying, and freeing your own pages of memory in a module CSE 522 S – Advanced Operating Systems 15
- Slides: 15