Memory Management Basic memory management Swapping Virtual memory

Memory Management Ø Basic memory management Ø Swapping Ø Virtual memory Ø Page replacement algorithms Ø Modeling page replacement algorithms Ø Design issues for paging systems Ø Implementation issues Ø Segmentation 1

Memory Management • Ideally programmers want memory that is large, fast, non volatile • Memory hierarchy – small amount of fast, expensive memory – cache – some medium-speed, medium price main memory – gigabytes of slow, cheap disk storage larger • Memory manager handles the memory hierarchy faster Intel : 8 -, 16 -, 32 -bits MIPS: 32 - bit 32 KB to a few MB 128 MB to 1 GB 40 GB to 160 GB 2

Basic Memory Management 3

Basic Memory Management Memory management: (1) swapping and paging (2) without swapping and paging Monoprogramming without Swapping or Paging Model (a) was used on mainframes and minicomputers, and is rarely used any more. Model (b) is used on some palmtop computers and embedded systems. Model (c) was used by the early personal computers. The portion of the system in ROM is called BIOS (Basic Input Output System) Except on simple embedded systems, monoprogramming is hardly used anymore. 4

Multiprogramming with Fixed Partitions (a) separate input queues for each partition (b) Single input queue 5

• (-) multiple input queues – queue for a large partition is empty but queue for a small partition is full – since the partitions are fixed, any space in a partition not used by a job is lost • single input queue: whenever a partition becomes free, the job closest to the front of the queue that fits in it could be loaded into the empty partition and run • different strategy: since it is undesirable to waste a large partition on a small job, search the whole input queue whenever a partition becomes free and pick the largest job that fits the partition. 6

Swapping 9

Swapping Two general approaches to memory management Ø Swapping: Method of copying a process’s memory contents to secondary storage, removing the process from the memory and allocating the new free memory to a new process, running it for a while, then putting it back on disk. Ø Virtual memory: Capability of operating systems that enables programs to address more memory locations than are actually provided in main memory. Virtual memory systems help remove much of the burden of memory management from programmers, freeing them to concentrate on application development Sec. 4. 3. 10

Memory allocation time l Swapping system: Ø The number of processes in memory varies dynamically. Ø Locations of processes in memory vary dynamically. Ø Size of the partitions varies dynamically. l Memory Compaction: When swapping creates multiple holes in memory, it is possible to combine them all into one big one by moving all the processes downward as far as possible. Ø Usually not done because it requires a lot of CPU time. 11

How much memory should be allocated for a process when it is created or swapped? Ø If processes are created with a fixed size that never change, then the allocation is simple: the OS allocates exactly what is needed, no more and no less. Ø If processes’ data segments can grow, a problem occurs whenever a process tries to grow. 12

Allocation space for a growing data (a)Allocating space for growing data segment If the hole between processes A and B runs out, A or B will have to be moved to a hole with enough space, swapped out of the memory until a large enough hole can be created, or killed. (b)Allocating space for growing stack & data segment If the hole between stack segment and data segment runs out, the process will have to be moved to a hole with enough space, swapped out of the memory until a large enough hole can be created, or killed. 13

• Two ways to keep track of memory usage – bit maps – lists 14

Memory Management with Bit Maps • Memory is divided up into allocation units, the size of unit may be as small as a few words as large as several kilobytes. • Part of memory with 5 processes, 3 holes – tick marks show allocation units – shaded regions are free 15

Trade-off: Ø The smaller the allocation unit, the larger the bitmap. Ø If the allocation unit is chosen large, the bitmap will become smaller, but the memory may be wasted in the last unit of the process if the process size is not an exact multiple of the allocation unit. Main problem: Ø When it has been decided to bring a k-unit process into memory, the memory manager must search the bitmap to find a run of k consecutive 0 bits in the map. Searching a bitmap for a run of a given length is a slow operation. 16

Memory Management with Linked Lists Linked list of allocated and free memory segments • The segment list is kept sorted by address. Sorting this way has advantage that when a process terminates or is swapped out, updating the list is straightforward. 17

(a) (b) (c) (d) Updating the list requires replacing a P with H. Two entries are coalesced into one, and the list becomes one entry shorter. The same with (b). Three entries are merged and two items are removed from the list. 18

Algorithms to allocate memory for a newly created process Assume that the memory manager knows how much memory to allocate. l First fit : The memory manager scans along the list of segments until it finds a hole that is big enough. The hole is then broken up into two pieces, one for the process and one for the unused memory. Ø It is a fast algorithm because it searches as little as possible. l Next fit : It works the same way as first, except that it keeps track of where it is whenever it finds a suitable hole. The next time it is called to find a hole, it starts searching the list from the place where it left off last time. Ø Simulations (Bays, 1977) show that it gives slightly worse performance than first fit. l Best fit: It searches the entire list and takes the smallest hole that is adequate. Ø It is slower than first fit. 19

l Worst fit : To get around the problem of breaking up nearly exact matches into a process and tiny hole, it always takes the largest available hole, so that the hole broken off will be big enough to be useful. Ø Simulation has shown that the worst fit is not a very good idea either. l Quick fit : It maintains separate lists for some of the more common sizes requested. Ø e. g. a table with n entries, in which the first entry is a pointer to the head of a list of 4 -KB holes, the second entry is the a pointer to a list of 8 -KB holes, the third entry a pointer to 12 -KB holes. Ø Finding a hole of required size is fast. Ø It has the same disadvantage as all schemes that sort by hole size, when a process terminates or is swapped out, finding its neighbor to see if a merge is possible is expensive. 20

Virtual Memory 21

Virtual Memory l Virtual memory: Capability of operating systems that enables programs to address more memory locations than are actually provided in main memory. Virtual memory systems help remove much of the burden of memory management from programmers, freeing them to concentrate on application development (Devised by Fotheringham, 1961) l Basic idea: the combined size of a program, data, and stack may exceed the amount of physical memory available for it. OS keeps those parts of the program currently in use in main memory, and the rest on disk l e. g. 16 -MB program can run on a 4 -MB machine by carefully choosing which 4 -MB to keep in memory at each instant, with pieces of program being swapped between disk and memory as needed. 22

Paging • Paging: Virtual memory organization technique that divides an address space into fixed blocks of contiguous address. When applied to a process’s virtual address space, the blocks are called pages, which store process data and instructions. When applied to main memory, the blocks are called page frames. 23

l Virtual address: Program-generated address (using indexing, base registers, segment registers and other ways). l Virtual address space: formed by all virtual address. Pentium II pro: 36 bits address: 236 = 64 GB l Memory management unit (MMU): a chip or collection of chips that maps the virtual addresses onto the physical memory addresses 24

Example of how the mapping works. ØVirtual addresses: 16 -bit (0 – 64 KB) ØPhysical memory: 64 KB ØUser program can be up to 64 KB, but it cannot be loaded into memory entirely and run. ØThe virtual address space is divided into units called pages. ØThe corresponding units in physical memory are called page frames. ØThe pages and frame pages are always the same size. 4 KB (512 B – 64 KB in real system) Ø 8 frame pages, 16 virtual pages e. g. MOV REG, 0 it is transformed into (by MMU) MOV REG, 8192 25

e. g. MOV REG, 8192 is transformed into MOV REG, 24576 In the actual hardware, a Present/absent bit keeps track of which pages are physically present in memory. (2456728671) (8192 -12287) (4196 -8191) (0 -4095) 26

Page fault: Fault that occurs as the result of an error when a process attempts to access a nonresident page, in which case the OS can load it from disk. e. g. MOV REG, 32780 (12 -th byte within virtual page 8) (1) MMU notices that the page is unmapped and causes CPU to trap to OS. (2) OS picks a little-used page frame and writes back to the disk. (3) Then it fetches the page just referenced into frame page just freed. (4) Change the map and restart the trapped instruction. 27

Page Tables Page table: Table that stores entries that map page numbers to page frames. A page table contains an entry for each of a process’s virtual pages. e. g. 16 -bit address: High-order 4 bits: virtual page number. Low-order 12 bits: offset 8196 is transformed into 24580 by MMU. Internal operation of MMU with 16 4 KB pages 28

l The purpose of page table is to map virtual pages onto page frames. l Two major issues must be faced: (1) The page table can be extremely large. e. g. a computer uses 32 -bit virtual addresses, page size: 4 KB Page number = 232/ 212 = 220 (1 million) Remember that each process needs its own page table because it has its own virtual address space. (2) The mapping must be fast. The virtual-to-physical mapping must be done on every memory reference. A typical instruction has an instruction word, and often a memory operand as well. Consequently, it is necessary to make 1, 2, or sometimes more page table reference per instruction. 29

Hardware solutions: l Simplest design: one page table consisting of an array of fast hardware registers, with one entry for each virtual page, indexed by virtual page number. Ø Advantage: straightforward, and requires no memory reference. Ø Disadvantage: expensive (if the page table is large) l Page table entirely in main memory, and one hardware register that points to the start of the page table Ø Advantage: allows the memory map to be changed at a context switch by reloading one register. Ø Disadvantage: requires one or more memory references to read page table entries during the execution of each instruction. l Variations of the two approaches 30

Second-level Page tables Multilevel Page Tables To get around the problem of having to store huge page tables in memory all the. Second-level page tables time. 32 -bit virtual address PT 1: 10 bits, PT 2: 10 bits Offset: 12 bits Top-level page table (Page size: 4 KB ) Page number: 220 The secret to the multilevel page table method is to avoid keeping all tables in memory all the time. e. g. a process needs 12 Mbytes, 4 MB for text, the next 4 MB for data, and the top 4 MB for stack. Only 4 page tables are actually needed: top-level table, second level tables for 0 to 4 M, 4 M to 8 M, and top 4 M. e. g. Virtual address = 0 x 00402004, then PT 1=1, PT 2=2, Offset=4 31

Structure of a Page Tables Entry l The exact layout of an entry is highly machine dependent, but the kind of information present is roughly the same. l The size varies from computer to computer, but 32 bits is a common size. l Page frame number: the goal of the page mapping is to locate this value. l Present/absent bit: If this bit is 1, the entry is valid and can be used. If it is 0, the virtual page to which the entry belongs is not currently in memory. l Modified and Referenced bits: keep track of page usage. When a page is written to, the hardware automatically sets the modified bit. If the page in it has been modified, it must be written back to the disk. Modified bit is sometimes called dirty bit. The reference bit is set whenever a page is referenced. l Caching disabled bit: allows caching to be disabled for the page. 32

TLBs – Translation Lookaside Buffers l All paging schemes keep the page tables in memory => performance problems! l Most programs tend to make a large number of references to a small number of pages, and not the other way around l Solution: equip computers with a small hardware device for mapping virtual addresses to physical addresses without going through the page table l This device is called associative memory (AM) or translation lookaside buffer. It is usually inside the MMU and consists of a small number of entries (normally 32) 33

A TLB to speed up paging l When a virtual address is presented to the MMU for translation, the hardware first check to see if its virtual page number is present in TLB by comparing it to all the entries simultaneously. If a valid match is found and the access does not violate the protection bits, the page frame is taken directly from TLB, without going to the page table. l Hit ratio: fraction of memory references that can be satisfied from the TLBs. The higher the hit ratio, the better the performance. l When the virtual page number is not in TLB, the MMU detects the miss and does an ordinary page lookup. 34

Software TLB Management l Hardware TLB Management – MMU hardware recognizes the virtual memory has page table. TLB management and TLB fault handling are done by TLB. l Software TLB Management – Modern RISC computers do nearly all of these page management in software. – e. g. SPARC, MIPS, Alpha, and HP PA. – On these machines, TLB entries are explicitly loaded by the OS. When a TLB miss occurs, it just generates a TLB fault and tosses the problem to OS. The OS must find the page, remove an entry from the TLB, enter the new one, and restart the instruction that faulted. And, of course, all of this must be done in a handful of instructions because TLB misses occur much more frequently than page faults. – If TLB is reasonably large to reduce the miss rate, software management of TLB turns out to be acceptably efficient (Uhlig, 1994). – Main gain: simpler MMU, more area on CPU chip for cache and other features. 35

Inverted Page Tables l Today: 32 -bit virtual address space and physical memory, 4 Kbytes pages size => each process need 2 20 entries in its page table (PT) with 4 bytes per entry = 4 Mbytes / process and PT is large but manageable (multilevel paging schemes) l RISC chips with 64 -bit virtual address space? : – 64 -bit virtual address space >>>> physical memory – 64 -bit address space = 20 million terabytes – 4 Kbytes page size => 2 52 = 4 quadrillion PT entries => requires rethinking!!!!! • Solution: virtual address space immense, physical pages frames still manageable => inverted page table in this design, there is one entry per page frame in real memory, rather than one entry per page of virtual address space. E. g. with 64 -bit virtual addresses, a 4 -KB page, and 256 MB of RAM, and inverted page table only requires 65, 536 entries. The entry keeps track of which (process, virtual page) is located in the page frame. 36

All virtual pages currently in memory that have the same hash value are chained together l Comparison of a traditional page table with an inverted page table l IBM and HP workstations use inverted page tables. It will become more common as 64 -bit machines become wide-spread. 37

Page Replacement Algorithms 38

Page Replacement Algorithms • • Page fault => OS has to select a page for replacement Modified page => write back to disk Not modified page => just overwrite with new page How to decide which page should be replaced? – random – many algorithms take into account • usage • . . . 39

Optimal Page Replacement Algorithm What is optimal page replacement algorithm? Unrealizable page-replacement strategy that replaces the page that will not be used until furthest in the future. Ø Easy to describe - impossible to implement because OS cannot look into future Ø Useful to evaluate page replacement algorithms Ø Best (optimal) page replacement algorithm – page fault occurs, a set of pages is in memory – label all pages with the number of instructions that will be executed before this page will be used again in the future – replace the page with the highest number It is of no use in practical. 40

NRU(Not Recently Used) Page Replacement Algorithm What is NRU page replacement algorithm? Page replacement strategy that uses referenced bits and modified bits to replace page. • Status bits associated with each page – R: page referenced (read or written) – M: page modified (written) (dirty bit, dirty page) • Four classes: – class 0: not referenced, not modified Check in – class 1: not referenced, modified this order – class 2: referenced, not modified – class 4: referenced, modified • NRU removes a page at random from the lowest numbered nonempty class • Low overhead 41

FIFO Page Replacement Algorithm What is FIFO page replacement algorithm? It is a page replacement strategy that replaces the page that has been in memory longest. l l l OS maintains list of all pages currently in memory. Pages are stored in list by age. FIFO replaces oldest pages in case of page fault. Incurs low overhead, but does not predict future page usage accurately. FIFO is rarely used in its pure form. Page loaded first Time 0 A 3 B 7 C 8 D 12 E 14 F 15 G 18 H Most recently loaded page 42

Second Chance Page Replacement Algorithm What is second chance page replacement algorithm? It is a variation of FIFO page replacement that uses the referenced bit and FIFO queue to determine which page to replace. If the oldest page’s referenced bit is off, it replace the page. Otherwise it turns off the referenced bit on the oldest page and moves it to the tail of FIFO queue, and examines the next page or pages until it locates a page with its referenced bit turned off. • R: referenced bit. • Second chance is a reasonable algorithm • But, inefficient because it is moving pages around on its list Page loaded first Time 0 A 3 B 7 C 8 D 12 E 14 F 15 G 18 H 20 A Most recently loaded page A is treated like newly loaded page 43

The Clock Page Replacement Algorithm When a page fault occurs, the page the arrow is pointing to is inspected. Action taken depends on the R bit R=0: evict page R=1: clear R & advance What is clock page replacement? It is a variation of second chance page replacement strategy that arranges the pages in a circular list instead of a linear list. • Pointer to the oldest page – R bit 0: page not referenced in last round => replace – R bit 1: page referenced in last round • set R bit to 0 • advance until first page with R = 0 is found – advance pointer to next entry in both cases 44

Least Recently Used (LRU) Page Replacement Algorithm What is LRU page replacement algorithm? Page-replacement strategy that replaces the page that has not been referenced for longest time. LRU generally predicts future page usage well but incurs significant overhead. (1) Linked list. It is expensive: maintaining the list is time consuming operation. (2) Implement with special hardware — a counter. Each page table entry must also have a filed large enough to contain the counter. (3) Another special hardware that can contain a matrix of n n bits, initially all 0. At any instant, the row whose value is lowest is the least recently used. 0 1 2 3 0 0 1 11 0 0 1 1 0 00 1 0 0 0 0 1 0 1 1 1 00 1 1 0 0 0 0 0 1 1 1 0 0 1 1 0 1 2 0 0 0 0 00 0 1 1 1 0 0 3 0 0 00 0 1 1 1 0 0 0 0 0 1 1 0 0 1 1 0 0 1 1 1 0 0 0 0 1 0 0 0 1 1 1 0 0 0 1 0 0 Pages referenced in this order: 0 1 2 3 2 1 0 3 2 3 45

Simulating LRU in Software Previous LRU algorithms are realizable in principle if machines have this hardware. They are no use to OS designer who is making a system for a machine that does not have this hardware. Solution: NFU (Not Frequently Used) algorithm: It requires a software counter associated with each page, initially zero. At each clock interrupt, OS scans all pages in memory. For each page, the R bit (0 or 1) is added to the counter. —Main problem of NFU algorithm: it never forget anything. Aging: Modifies NFU algorithm as follows, and makes it able to simulate LRU quite well. (1) The counters are each shifted right 1 bit before R bit is added in; (2) The R bit is added to the leftmost, rather than the rightmost. 46

The aging algorithm simulates LRU in software Note 6 pages for 5 clock ticks, (a) – (e) In practice, 8 bits is enough if a clock tick is around 20 msec. 47

The Working Set Page Replacement Algorithm W(k, t) k Working set: the set of pages that a process is currently using. k: most recent memory reference t: time w(k, t): the size of the working set at time, t 48

page span = current virtual time – time of last use : predetermined page span The working set page replacement algorithm: l The hardware is assumed to set R and M bits. l A periodic clock interrupt is assumed to cause software to run that clears R bit on every clock tick. l On page every fault, the page table is scanned to look for a suitable page to evict. 49

algorithm but also uses the working set information. page span = current virtual time – time of last use : predetermined page span. 50

Review of Page Replacement Algorithms 51

Segmentation 52

Segmentation Problem in one-dimensional address space with growing tables Ex. A compiler has following tables (1) Source text (2) Symbol table (3) Constant table (4) Parse tree (5) Stack Problem: one table may bump into another Solution: To provide the machine with many completely independent address spaces, called sgements. 53

Segment: Variable-size set of contiguous addresses in a process’s virtual address space that is managed as one unit. A segment is typically the size of an entire set of similar items, such as a set of instructions in a procedure or the contents of an array, which enables the system to protect such items with fine granularity using appropriate access rights. – two or more separate/independent virtual address spaces growing/shrinking – different kinds of protection are possible Two-part address (n, k): – n: address number (which segment) – k: address within segment Segmentation also facilitates sharing procedures or data between several processes – e. g. shared library 54

• Segmented memory allows each table to grow or shrink independently of other tables 55

Comparison of paging and segmentation 56

Implementation of Pure Segmentation The implementation of segmentation differs from paging in an essential way: Pages are fixed size and segments are not. (a)-(d) Development of checkerboarding (e) Removal of the checkerboarding by compaction External fragment (or checkerboarding): After the system has been running for a while, memory will be divided up into a number of chunks, some containing segments and some containing holes. This phenomena is called external fragment. 57

Segmentation with Paging: MULTICS (MULTiplexed Information and Computer Service) : One of the first operating systems to implement virtual memory. Developed by MIT, GE and Bell Laboratories as the successors to MIT’s CTSS (Compatible Time Sharing System). Ken Thompson, one of the computer scientists at Bell Labs who had worked on MULTICS project, wrote a strippeddown, one-user version of MULTICS. This work later developed into UNIX. 58

Segmentation with Paging • Many large segments > main memory size => paging • MULTICS – Honeywell 6000 machines + descendents – per program: virtual memory of max. size 218 = 256 K segments (max. size 64 K 36 -bit word long) – Treat each segment as a virtual memory and to page it. – segment table + page tables – 16 -word high speed TLB 59

• Descriptor segment points to page tables 64 K 60

A 34 -bit MULTICS virtual address 61

Memory reference Ø Conversion of a 2 -part MULTICS address into a main memory address Ø Problem: program would not run very fast. Ø Solution: 16 -word TLB 62

Simplified version of the MULTICS TLB (Existence of 2 page sizes makes actual TLB more complicated) 63

Segmentation with Paging: The Intel Pentium • MULTICS: – Both segmentation and paging – 256 K independent segments, each up to 64 K 36 -bit words • Intel Pentium – Both segmentation and paging – 16 K independent segments, each up to 1 billion 32 -bit words – Each program has its own LDT (Local Descriptor Table). LDT describes segments local to each program, including its code, data, stack, and so on. – A single GDT (Global Descriptor Table) shared by all programs on the computer. GDT describes system segments including the OS its self. 64

To access a segment, a Pentium program first loads a selector for that segment into one of the machine’s 6 segment register. CS: holds the selector for code segment DS: holds the selector for data segment A Pentium selector Specify LDT or GDT entry number. Theses tables are restricted to hold 8 K segment descriptors. 65

At the time a selector is loaded into a segment register, the corresponding descriptor is fetched from the LDT or GDT and stored in microprogram registers, so it can be accessed quickly. Pentium code segment descriptor (Data segments differ slightly): 8 bytes 66

How to convert (selector, offset) pair to physical address ? (1) Find the descriptor corresponding to the selector. If the segment does not exist, or is currently paged out, a trap occurs. (2) Check the offset is beyond the end of the segment, in which case a trap occurs. (3) If G(granularity)=0, limit field (20 bits) is the exact segment size, up to 1 MB. If G=0, limit field gives the segment size in pages instead of bytes. Pentium page size is fixed as 4 KB, 20 bits are enough for segments up to 232 bytes. (4) (3) Assuming that the segment is in memory and the offset is in range, the Pentium then adds 32 -bit base field to offset to form linear address. (5) 32 -bit base is broken into 3 pieces all over descriptor for compatibility with 286 (base is 24 bits) (6) (4) If paging is disabled (by a bit in global control register), the linear address is interpreted as the physical address and sent to memory for read or write. This is a pure segmentation scheme. (7) (5) If paging is enabled, the linear address is interpreted as a virtual address and mapped onto physical address using page tables. Page size is 4 KB, a segment might contain 1 million pages. 67

Conversion of a (selector, offset) pair to a linear address 68

Each running program has a page directory consisting of 1 K 32 -bit entries. Located at an address pointed to by a global register. Each entry in this directory points to a table also containing 1 K 32 -bit entries. Mapping of a linear address onto a physical address 69

Page table entry: 32 bits each, 20 of which contains page frame number, remaining bits contains access and dirty bits, set by hardware, Single page table handles 4 MBytes of memory (1 K page frames, page size is 4 KB) To avoid making repeated reference to memory, the Pentium (like MULTICS) has a small TLB that directly maps the most recently used Dir-page combination onto physical address of the page frame. If some application does not need segmentation but is content with a single, paged, 32 -bit address, the model is possible. All segment registers can be set up with the same selector, whose descriptor has base=0 and limit set to maximum. In fact all current OSs for Pentium work this way. OS/2 was the only one that used full power of Intel MMU architecture. 70

Level Protection on the Pentium Ø Pentium supports 4 protection level. A running program is at a certain level indicated by 2 bits in PSW(processor status word). Ø Each segment in the system also has a level. 71