Memory management part 3 outline q Segmentation q

Memory management, part 3: outline q Segmentation q Case studies o MULTICS o Pentium o Unix o Linux o Windows Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 1

Segmentation q Several address spaces per process q a compiler needs segments for o o o source text symbol table constants segment stack parse tree compiler executable code q Most of these segments grow during execution symbol table Source source Text text constant table parse tree call stack Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 2

Users view of segments Symbol table Parse tree Source text Call stack Constants A segmented memory allows each table to grow or shrink independently of the other tables. Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 3

Segmentation - segment table Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 4

Segmentation Hardware Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 5

Segmentation vs. Paging consideration Paging Segmentation Need the program be aware of the technique ? no yes How many per-process virtual address spaces ? 1 many Can the total address space exceed physical memory ? yes Can procedures and data be distinguished ? no yes Sharing of procedures among users facilitated ? no yes Motivation for the technique Ben-Gurion University Get larger linear space, eliminate external fragmentation Programs and data in logical independent address spaces, sharing and protection made simpler Operating Systems, 2011, Danny Hendler and Amnon Meisels 6

Segmentation pros and cons q Advantages: o Growing and shrinking independently. o Sharing between processes simpler o Linking is easier o Protection easier q Disadvantages: o Pure segmentation --> external Fragmentation revisited o Segments may be very large. What if they don't fit into physical memory? Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 7

Segmentation Architecture q. Logical address composed of the pair <segment-number, offset> q. Segment table – maps to linear address space; each table entry has: o base – contains the starting linear address where the segment resides in memory. o limit – specifies the length of the segment. q. Segment-table base register (STBR) points to the segment table’s location in memory. q. Segment-table length register (STLR) indicates number of segments used by a program; segment number s is legal if s < STLR. Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 8

Segmentation Architecture (Cont. ) q Protection: each segment table entry contains: o validation bit = 0 illegal segment o read/write/execute privileges q Protection bits associated with segments; code sharing occurs at segment level. q Since segments vary in length, memory allocation is a dynamic storage-allocation problem (external fragmentation problem) Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 9

Sharing of segments Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 10

Segmentation with Paging q Segments may be too large q Cause external fragmentation q The two approaches may be combined: o Segment table. o Pages inside a segment. o Solves fragmentation problems. q Most systems today provide a combination of segmentation and paging Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 11

Memory management, part 3: outline q Segmentation q Case studies o MULTICS o Pentium o Unix o Linux o Windows Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 12

The MULTICS OS q Ran on Honeywell computers q Segmentation + paging q Up to 218 segments q Segment length up to 216 36 -bit words q Each program has a segments table (itself a segment) q Each segment has a page table Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 13

MULTICS data-structures 36 bits Segment 4 descriptor 18 bits Segment 3 descriptor Segment 2 descriptor Page 2 entry Page 1 entry Page 0 entry Page table for segment 3 Page table for segment 1 18 bits Segment 1 descriptor Segment 0 descriptor Process descriptor segment 18 bits 6 bits Main memory address of the page table 3 3 Segment length (in pages) Segment descriptor Page size: 0 – 1024 word 1 – 64 words 0 – paged 1 1 1 misc Unused Protection bits 1 – not paged Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 14

MULTICS memory reference procedure 1. Use segment number to find segment descriptor Segment table is itself paged because it may be large. The descriptorbase-register points to its page table. 2. Check if segment’s page table is in memory o if not a segment fault occurs o if there is a protection violation TRAP (fault) 3. page table entry examined, a page fault may occur. o if page is in memory the start-of-page address is extracted from page table entry 4. offset is added to the page origin to construct main memory address 5. perform read/store etc. Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 15

MULTICS Address Translation Scheme Segment number (18 bits) Page number (6 bits) Page offset (10 bits) Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 16

MULTICS TLB q Simplified version of the MULTICS TLB q Existence of 2 page sizes makes actual TLB more complicated Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 17

Memory management, part 3: outline q Segmentation q Case studies o MULTICS o Pentium o Unix o Linux o Windows Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 19

Pentium: Segmentation + paging q Segmentation with or without paging is possible q 16 K segments per process, segment size up to 4 G 32 -bit words q page size 4 K q A single global GDT, each process has its own LDT q 6 segment registers may store (16 bit) segment selectors: CS, DS, SS… q When the selector is loaded to a segment register, the corresponding descriptor is stored in microprogram registers 13 = 0 GDT/ 1 = LDT Privilege level (0 -3( 1 2 Index Pentium segment selector Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 20

Pentium- segment descriptors Pentium code segment descriptor. Data segments differ slightly Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 21

Pentium - Forming the linear address q Segment descriptor is in internal (microcode) register q If segment is not zero (TRAP) or paged out (TRAP) o Offset size is checked against limit field of descriptor o Base field of descriptor is added to offset (4 k page-size) Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 22

Intel Pentium address translation 10 10 12 Can cover up to 4 MB physical address space Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 23

Memory management, part 3: outline q Segmentation q Case studies o MULTICS o Pentium o Unix o Linux o Windows Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 24

UNIX process address space Process B Process A Stack pointer 20 K 8 K 0 BSS Init. Data Text OS Physical memory Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 25 20 K 8 K 0

Memory-mapped file Process B Process A Stack pointer Memory mapped file 20 K 8 K 0 Memory mapped file BSS Data Text OS Physical memory Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 26 20 K 8 K 0

Unix memory management sys calls q Not specified by POSIX q Common Unix system calls o s=brk(addr) – change data segment size. (addr sepcified the first address following new size) o a=mmap(addr, len, prot, flags, fd, offset) – map (open) file fd starting from offset in length len to virtual address addr (0 if OS is to set address) o s=unmap(addr, len) – unmap a file (or a portion of it) Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 27

Unix 4 BSD memory organization Main memory Core map entry Used when page frame is on free list Page frame 3 Index of next entry Index of previous entry Disk block number Page frame 2 Disk device number Page frame 1 Block hash code Page frame 0 Index into proc table Core map entries, one per page frame Text/data/stack Offset within segment Misc. Kernel Free Ben-Gurion University In transit Operating Systems, 2011, Danny Hendler and Amnon Meisels 28 Wanted Locked

Unix Page Daemon q It is assumed useful to keep a pool of free pages q freeing of page frames is done by a pagedaemon - a process that sleeps most of the time q awakened periodically to inspect the state of memory if less than ¼ 'th of page frames are free, then it frees page frames q this strategy performs better than evicting pages when needed (and writing the modified to disk in a hurry) q The net result is the use of all of available memory as page-pool q Uses a global clock algorithm – two-handed clock Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 29

Page replacement - Unix q a two-handed clock algorithm clears the reference bit first with the first hand frees pages with its second hand. It has the parameter of the “angle” between the hands - small angle leaves only “busy” pages o If page is referenced before 2’nd hand comes, it will not be freed Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 30

Page replacement – Unix, cont'd q if there is thrashing, the swapper process removes processes to secondary storage o Remove processes idle for 20 sec or more o If none – swap out the oldest process out of the 4 largest q Who get swapped back function of: o Time out of memory o size Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 31

Memory management, part 3: outline q Segmentation q Case studies o MULTICS o Pentium o Unix o Linux o Windows Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 32

Linux processes q Each process gets 3 GB virtual memory q Remaining 1 GB for kernel and page tables q Virtual address space composed of areas with same protection, paging properties (pageable or not, direction of growth) q Each process has a linked list of areas, sorted by virtual address (text, data, memory-mapped-files, …) Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 33

vm_area_struct Process virtual address space partitioned to areas struct vm_area_struct { unsigned long vm_start; unsigned long vm_end; unsigned long vm_flags; struct vm_area_struct*vm_next; /* plus some other fields */ Ben-Gurion University }; Operating Systems, 2011, Danny Hendler and Amnon Meisels 34

Linux page tables organization Directory Page middle directory Middle Page table Page Selected word Offset This is the situation in Alpha. In Pentium, the page middle directory is degenerated. Expanded to 4 -level indirect paging after Linux 2. 6. 10 Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 35

Linux main memory management q Kernel never swapped q The rest: user pages, file system buffers, variable-size device drivers q The buddy algorithm is used. In addition: o Linked lists of same-size free blocks are maintained o To reduce internal fragmentation, a second memory allocation scheme (slab allocator) manages smaller units inside buddy-blocks q Demand paging (no pre-paging) q Dynamic backing store management Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 36

Linux page replacement algorithm q Variant of clock algorithm q Based on aging, pages are in either active or inactive list Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 37

Memory management, part 3: outline q Segmentation q Case studies o MULTICS o Pentium o Unix o Linux o Windows Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 38

Win 2000: virtual address space q Virtual address space layout for 3 user processes q White areas are private per process q Shaded areas are shared among all processes What are the pros/cons of mapping kernel area into process address space? Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 39

Win 2000: memory mngmt. concepts q Each virtual page can be in one of following states: o Free/invalid – Currently not in use, a reference causes access violation o Committed – code/data was mapped to virtual page o Reserved – allocated to thread, not mapped yet. When a new thread starts, 1 MB of process space is reserved to its stack o Readable/writable/executable q Dynamic (just-in-time) backing store management o Improves performance of writing modified data in chunks o Up to 16 pagefiles q Supports memory-mapped files q Can use 4 K or 4 M pages q Executable access pattern recorded for Super. Fetch prepaging Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 40

Win 2000: page replacement alg. q Processes have working sets defined by two parameters - the minimal and maximal # of pages q the WS of processes is updated at the occurrence of each page fault (i. e. the data structure WS) o PF and WS < Min add to WS o PF and WS > Max remove from WS q If a process thrashes, its working set size is increased q Memory is managed by keeping a number of free pages, which is a complex function of memory use, at all times q when the balance-set-manager is run (every second) and it needs to free pages o surplus pages (to the WS) are removed from a process (large background before small foreground…) o Pages `age-counters’ are maintained (on a multi-processor refs bits don’t work since they are local…) Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 41

Physical Memory Management (1( Various page lists and transitions between them Ben-Gurion University Operating Systems, 2011, Danny Hendler and Amnon Meisels 43