Memory Organization 1 MEMORY ORGANIZATION Memory Hierarchy Main

Memory Organization 1 MEMORY ORGANIZATION • Memory Hierarchy • Main Memory • Auxiliary Memory • Associative Memory • Cache Memory • Virtual Memory • Memory Management Hardware calab. kaist. ac. kr/~hyoon/courses/cs 311_2006/Ch 11. ppt Computer Organization Computer Architectures Lab

Memory Organization 2 Memory Hierarchy MEMORY HIERARCHY Memory Hierarchy is to obtain the highest possible access speed while minimizing the total cost of the memory system Auxiliary memory Magnetic tapes I/O processor Main memory CPU Cache memory Magnetic disks Register Cache Main Memory Magnetic Disk Magnetic Tape Computer Organization Computer Architectures Lab

Memory Organization 3 Main Memory MAIN MEMORY RAM and ROM Chips Typical RAM chip Chip select 1 Chip select 2 Read Write 7 -bit address CS 1 CS 2 0 0 0 1 1 0 1 0 1 1 CS 2 RD WR AD 7 RD x x 0 0 1 x WR x x 0 1 x x 128 x 8 RAM 8 -bit data bus Memory function Inhibit Write Read Inhibit State of data bus High-impedence Input data to RAM Output data from RAM High-impedence Typical ROM chip Chip select 1 Chip select 2 9 -bit address Computer Organization CS 1 CS 2 AD 9 512 x 8 ROM 8 -bit data bus For the same size of chip, it is possible to have more bits of ROM than a RAM, because the internal binary cells in ROM occupy less space than in RAM. Computer Architectures Lab

Memory Organization 4 Main Memory MEMORY ADDRESS MAP Address space assignment to each memory chip Example: 512 bytes RAM and 512 bytes ROM Component RAM RAM ROM 1 2 3 4 Hexa address 0000 - 007 F 0080 - 00 FF 0100 - 017 F 0180 - 01 FF 0200 - 03 FF Address bus 10 9 0 0 1 1 x 8 7 6 5 4 3 2 1 0 1 x x x x x x x x x Memory Connection to CPU - RAM and ROM chips are connected to a CPU through the data and address buses - The low-order lines in the address bus select the byte within the chips and other lines in the address bus select a particular chip through its chip select inputs Computer Organization Computer Architectures Lab

Memory Organization 5 Main Memory CONNECTION OF MEMORY TO CPU CS 1 CS 2 128 x 8 RD RAM 2 WR AD 7 CS 1 CS 2 128 x 8 RD RAM 3 WR AD 7 CS 1 CS 2 128 x 8 RD RAM 4 WR AD 7 1 - 7 8 9 Computer Organization CS 1 CS 2 512 x 8 AD 9 ROM Data CS 1 CS 2 128 x 8 RD RAM 1 WR AD 7 Data Decoder 3 2 1 0 Data bus Data RD WR Data 7 -1 Data Address bus 16 -11 10 9 8 Computer Architectures Lab

Memory Organization 6 Associative Memory ASSOCIATIVE MEMORY - Accessed by the content of the data rather than by an address - Also called Content Addressable Memory (CAM) Hardware Organization Argument register(A) Key register (K) Match register Input Read Write Associative memory array and logic M m words n bits per word Out Put - Compare each word in CAM in parallel with the content of A(Argument Register) - If CAM Word[i] = A, M(i) = 1 - Read sequentially accessing CAM for CAM Word(i) for M(i) = 1 - K(Key Register) provides a mask for choosing a particular field or key in the argument in A (only those bits in the argument that have 1’s in their corresponding position of K are compared) Computer Organization Computer Architectures Lab

Memory Organization 7 Example A K 101 111100 111 000000 Word 1 100 111100 Word 2 101 000001 Computer Organization no match Computer Architectures Lab

Memory Organization 8 Associative Memory ORGANIZATION OF CAM A 1 Aj An K 1 Kj Kn Word 1 C 1 j C 1 n M 1 Word i Ci 1 Cij Cin Mi Word m Cm 1 Cmj Cmn Mm Bit 1 Bit j Bit n Internal organization of a typical cell Cij Aj Input Kj Write R Read S F ij Match logic To M i Output Computer Organization Computer Architectures Lab

Memory Organization 9 Match Logic • The match logic for each word can be derived from the comparison algorithm for two binary numbers. First we neglect the key bits and compare the argument in A with the bits stored in cell of the words. The equality of two words can be expressed logically by the Boolean function xj = Aj. Fij + Aj’Fij’ Where xj = 1 if the pair of the bits in position j are equal , otherwise xj = 0. For a word I to be equal to the argument in A we must have all xj variables equal to 1. This the condition for setting the corresponding match bit Mi to 1. The Boolean function for this condition is Mi = x 1 x 2 x 3. . . ……. . xn Computer Architectures Lab Computer Organization

Memory Organization 10 Match Logic Now we include the key bit kj in the comparison logic. The requirement is that if kj = 0 , the corresponding bits Aj and Fij need no comparison. Only when Kj = 1 must be compared. xj if Kj = 1 xj + Kj’ = 1 if Kj = 0 A term (xj + Kj’) will be in 1 state if its pair of bits is not compared. . This is because each term is ANDed with all other terms so that an output of 1 will have no effect. Now the match logic for word I : Mi = (x 1 +K 1’) (x 2 +K 2’) (x 3 +K 3’). . (xn +Kn’) It can be expressed as ∏ is a product n symbol designing the AND operation Mi = ∏ (Aj. Fij + Aj’Fij’ + Kj’) of all n terms j=1 Computer Organization Computer Architectures Lab

Memory Organization 11 Associative Memory MATCH LOGIC K 1 A 1 F'i 1 F i 1 K 2 A 2 F'i 2 F i 2 Kn. . F'in An F in Mi Computer Organization Computer Architectures Lab

Memory Organization 12 Cache Memory CACHE MEMORY Locality of Reference - The references to memory at any given time interval tend to be confined within a localized areas - This area contains a set of information and the membership changes gradually as time goes by - Temporal Locality The information which will be used in near future is likely to be in use already( e. g. Reuse of information in loops) - Spatial Locality If a word is accessed, adjacent(near) words are likely accessed soon (e. g. Related data items (arrays) are usually stored together; instructions are executed sequentially) Cache - The property of Locality of Reference makes the Cache memory systems work - Cache is a fast small capacity memory that should hold those information which are most likely to be accessed Main memory 32 K x 12 Computer Organization CPU Cache memory 512 X 12 Computer Architectures Lab

Memory Organization 13 Cache Memory PERFORMANCE OF CACHE Memory Access All the memory accesses are directed first to Cache If the word is in Cache; Access cache to provide it to CPU If the word is not in Cache; Bring a block (or a line) including that word to replace a block now in Cache - How can we know if the word that is required is there ? - If a new block is to replace one of the old blocks, which one should we choose ? Performance of Cache Memory System Hit Ratio - % of memory accesses satisfied by Cache memory system Te: Effective memory access time in Cache memory system Tc: Cache access time Tm: Main memory access time Te = Tc + (1 - h) Tm Example: Tc = 0. 4 s, Tm = 1. 2 s, h = 0. 85% Te = 0. 4 + (1 - 0. 85) * 1. 2 = 0. 58 s Computer Organization Computer Architectures Lab

Memory Organization 14 Cache Memory MEMORY AND CACHE MAPPING - ASSOCIATIVE MAPPLING Mapping Function Three types of mapping procedures are of practical interested when considering the organization of cache memory: Associative mapping Direct mapping Set-associative mapping Associative Mapping The associative memory stores both the address and content (data) of the memory word. This permits any location in cache to store any word from main memory. CPU address (15 bits) Argument register CAM Computer Organization Address Data 01000 02777 22235 3450 6710 1234 All numbers are in octal Computer Architectures Lab

Memory Organization 15 Cache Memory MEMORY AND CACHE MAPPING - DIRECT MAPPING - Each memory block has only one location to load in Cache - Mapping Table is made of RAM instead of CAM - n-bit memory address consists of 2 parts; k bits of Index field and n-k bits of Tag field - n-bit addresses are used to access main memory and k-bit Index is used to access the Cache Addressing Relationships Tag(6) Index(9) 00 000 32 K x 12 000 Main memory Address = 15 bits Data = 12 bits Direct Mapping Cache Organization Memory address 00000 Memory data 1220 00777 01000 2340 3450 01777 02000 4560 5670 02777 6710 Computer Organization 77 777 Index address 000 777 512 x 12 Cache memory 777 Address = 9 bits Data = 12 bits Cache memory Tag 00 Data 1220 02 6710 Computer Architectures Lab

Memory Organization 16 Cache Memory DIRECT MAPPING Operation - CPU generates a memory request with (TAG; INDEX) - Access Cache using INDEX ; (tag; data) Compare TAG and tag - If matches -> Hit Provide Cache[INDEX](data) to CPU - If not match -> Miss Take the word from main memory, also bring the word in cache. Direct Mapping with block size of 8 words Index tag data 000 007 010 017 01 01 3450 6578 Block 63 770 777 02 02 6710 Block 1 Computer Organization 6 Tag 6 Block 3 Word INDEX Computer Architectures Lab

Memory Organization 17 Cache Memory MEMORY AND CACHE MAPPING - SET ASSOCIATIVE MAPPING - Each word of cache can store two or more words of memory under the same index address. Set Associative Mapping Cache with set size of two : Each Tag require 6 bits and each data word has 12 bits so the word length is 2(6+12)=36 bits. [so size will be 512 X 36]. So It can accommodate 1024 words Of Main memory. Computer Organization Index Tag Data 000 01 3450 02 5670 777 02 6710 00 2340 Computer Architectures Lab

Memory Organization 18 Cache Memory BLOCK REPLACEMENT POLICY Many different block replacement policies are available: Random replacement First in First out (FIFO) replacement Least recently used (LRU) replacement Computer Organization Computer Architectures Lab

Memory Organization 19 Virtual Memory Give the programmer the illusion that the system has a very large memory, even though the computer actually has a relatively small main memory Address Space (Logical) and Memory Space (Physical): A virtual memory system provides a mechanism for translating program-generated addresses into correct main memory locations (by using mapping tables). address space virtual address (logical address) address generated by programs Computer Organization memory space Mapping physical address actual main memory address Computer Architectures Lab

Memory Organization 20 Virtual Memory VIRTUAL MEMORY Address Mapping Memory Mapping Table for Virtual Address -> Physical Address A mapping table may be stored in a separate memory or in main memory. Virtual address generated by the program is matched with the mapping table , Then actual physical address is send to the memory address register. Virtual address register Memory mapping table Memory table buffer register Computer Organization Main memory address register Physical Address Main memory buffer register Computer Architectures Lab

Memory Organization 21 Virtual Memory ADDRESS MAPPING Address Space and Memory Space are each divided into fixed size group of words called blocks or pages 1 K words group Address space N = 8 K = 213 Page 0 Page 1 Page 2 Page 3 Page 4 Page 5 Page 6 Page 7 Memory space M = 4 K = 212 Block 0 Block 1 Block 2 Block 3 Organization of memory Mapping Table in a paged system Page no. 1 0 1 Table address Memory page table Virtual address Presence bit 000 001 010 011 100 101 110 111 11 00 01 10 01 Computer Organization Line number 0 1 0 1 0 0 1 1 0 01 0101010011 Main memory address register Main memory Block 0 Block 1 Block 2 Block 3 MBR 1 Computer Architectures Lab

Memory Organization 22 Virtual Memory ASSOCIATIVE MEMORY PAGE TABLE Assume that Number of Blocks in memory = m Number of Pages in Virtual Address Space = n Page Table - Straight forward design -> n entry table in memory Inefficient storage space utilization <- n-m entries of the table is empty - More efficient method is m-entry Page Table made of an Associative Memory m words; (Page Number: Block Number) Virtual address Page no. 1 0 1 Line number 1 1 1 0 0 0 1 0 1 1 1 0 Page no. Computer Organization Argument register Key register Associative memory Block no. Computer Architectures Lab

Memory Organization 23 Virtual Memory PAGE FAULT 1. CPU reference any page, in memory page table the presence bit is invalid. 3 Page is on backing store 2 trap OS 2. Trap to the Operating system. 3. OS goes to the backing storage device where page is stored. 1 Reference LOAD M 4. OS bring the page in main empty memory frame (block) , if any frame is not empty then replace any frame (by using replacement algo. ) 0 6 restart instruction 4 5 reset page table free frame bring in missing page main memory 5. Reset the page table. 6. CPU restart the execution. Computer Organization Computer Architectures Lab

Memory Organization 24 Virtual Memory PAGE REPLACEMENT 1. Find the location of the desired page on the backing store 2. Find a free frame - If there is a free frame, use it - Otherwise, use a page-replacement algorithm to select a victim frame - Write the victim page to the backing store 3. Read the desired page into the (newly) free frame 4. Restart the user process valid/ frame invalid bit 2 change to f 0 v i invalid 4 v f reset page table for new page table swap out 1 victim page victim 3 swap desired page in backing store physical memory Computer Organization Computer Architectures Lab

Memory Organization 25 Virtual Memory PAGE REPLACEMENT ALGORITHMS FIFO Reference string 7 0 1 7 2 7 0 0 3 2 0 1 7 0 1 0 2 3 1 4 2 3 0 2 4 3 0 4 2 3 4 2 0 3 2 1 2 0 1 3 0 2 3 0 1 7 0 1 2 0 7 1 2 1 7 0 2 7 0 1 Page frames FIFO algorithm selects the page that has been in memory the longest time Using a queue - every time a page is loaded, its identification is inserted in the queue Easy to implement May result in a frequent page fault Optimal Replacement (OPT) - Lowest page fault rate of all algorithms Replace that page which will not be used for the longest period of time Reference string 7 0 7 1 7 0 2 7 0 1 0 2 0 1 3 0 2 0 3 4 2 2 4 3 3 0 3 2 1 2 2 0 1 7 0 1 Page frames Computer Organization Computer Architectures Lab

Memory Organization 26 Virtual Memory PAGE REPLACEMENT ALGORITHMS LRU - OPT is difficult to implement since it requires future knowledge - LRU uses the recent past as an approximation of near future. Replace that page which has not been used for the longest period of time Reference string 7 0 1 2 7 7 0 1 0 2 0 1 3 0 2 0 3 4 2 4 0 3 3 4 0 2 0 4 3 2 3 0 3 2 2 1 3 2 0 1 1 0 2 7 0 1 1 0 7 Page frames MRU (Most Recently used): Replace the page which is used most Recently. Computer Organization Computer Architectures Lab