Advanced Computer Architecture Module 2 Processors and Memory

Advanced Computer Architecture Module - 2 Processors and Memory Hierarchy Book: “Advanced Computer Architecture – Parallelism, Scalability, Programmability”, Hwang & Jotwani Presented By Dr. Vandana Jha

In this chapter… • • Design Space of Processors Superscalar and Vector Processors Memory Hierarchy Technology Virtual Memory Technology Dr. Vandana Jha, Department of CSE, KSSEM 2

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • Processor families mapped onto a coordinated space of clock rate v/s CPI o Clock Rates moved from lower to higher speeds o CPI rate is lowered • Broad Categorization o CISC o RISC Dr. Vandana Jha, Department of CSE, KSSEM 3

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • The Design Space o o o CISC Computers RISC Computers Superscalar Processors VLIW Processors Vector Supercomputers Dr. Vandana Jha, Department of CSE, KSSEM 4

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • Instruction Pipelines o o o o Instruction Cycle Phases Pipeline and Pipeline Cycle Instruction issue Latency Instruction Issue Rate (degree of superscalar processor) Simple Operation Latency Resource Conflicts Base scalar processor Dr. Vandana Jha, Department of CSE, KSSEM 5

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 6

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 7

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 8

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 9

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 10

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 11

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 12

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 13

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 14

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • Vector Processors o Memory to memory VP • Memory based instructions • Longer instructions • Instructions include memory address o Register to register VP • Shorter Instructions • Vector register files Dr. Vandana Jha, Department of CSE, KSSEM 15

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • Vector Processors o Vector Instructions (Register to register) • Binary Vector V 1 o V 2 V 3 • Scaling s 1 o V 1 V 2 • Binary Reduction V 1 o V 2 s 1 • Vector Load M(1: n) V 1 • Vector Store V 1 M(1: n) • Unary Vector o V 1 V 2 • Unary Reduction o V 1 s 1 Dr. Vandana Jha, Department of CSE, KSSEM 16

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 17

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • Vector Processors o Vector Instructions (Memory to memory) • M 1(1: n) o M 2(1: n) M 3(1: n) • s 1 o M 1(1: n) M 2(1: n) • o M 2(1: n) M(1: n) • M 1(1: n) o M 2(1: n) M(k) o Vector Pipelines • Symbolic Processors o Prolog Processors, Lisp Processors or symbolic manipulators. o Deals with logic programs, symbolic lists, objects, scripts, productions systems, semantic networks, frames and artificial neural networks. Dr. Vandana Jha, Department of CSE, KSSEM 18

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors Dr. Vandana Jha, Department of CSE, KSSEM 19

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • Symbolic Processors o Attributes & Characteristics of Symbolic Processing • Knowledge Representation o Lists, relational databases, semantic nets, frames etc. • Common Operations o Search, sort, pattern matching, filtering etc. • Memory Requirements o Large memory with intensive access pattern, content-based • Communication Patterns o Varying traffic size, granularity and format of messages Continued… Dr. Vandana Jha, Department of CSE, KSSEM 20

ADVANCED PROCESSOR TECHNOLOGY Design Space of Processors • Symbolic Processors o Attributes & Characteristics of Symbolic Processing • Properties of Algorithms o Non-deterministic, parallel and distributed computations • I/O Requirements o User-guided programs, intelligent person-machine interfaces • Architecture Features o Parallel update of knowledge bases, dynamic load balancing, dynamic memory allocation, hardware based garbage collection etc. Dr. Vandana Jha, Department of CSE, KSSEM 21

ADVANCED PROCESSOR TECHNOLOGY Memory Hierarchy Technology • Memory Hierarchy - Need & Significance Dr. Vandana Jha, Department of CSE, KSSEM 22

ADVANCED PROCESSOR TECHNOLOGY Memory Hierarchy Technology • Memory Hierarchy o Parameters • Access time • Memory size • Cost per byte • Transfer bandwidth • Unit of transfer o Properties • Inclusion • Coherence • Locality Dr. Vandana Jha, Department of CSE, KSSEM 23

ADVANCED PROCESSOR TECHNOLOGY Memory Hierarchy Technology • Memory Hierarchy o Parameters • T(i): Access time (round-trip time from CPU to i-th level memory) o T(i-1) < T(i) <T(i+1) • S(i): Memory size (number of bytes or words in i-th level memory) o S(i-1) < S(i+1) • C(i): Cost per byte (per byte cost of i-th level memory; total cost estimated by C(i)*S(i)) o C(i-1) > C(i+1) • B(i): Transfer bandwidth (rate at which information is transferred between adjacent levels) o B(i-1) > B(i+1) • X(i): Unit of transfer (grain size for data transfer between levels I and i+1) o X(i-1) < X(i+1) Dr. Vandana Jha, Department of CSE, KSSEM 24

ADVANCED PROCESSOR TECHNOLOGY Memory Hierarchy Technology • Memory Hierarchy o Properties • Inclusion Property o M 1 ⊂ M 2 ⊂ M 3 ⊂. . ⊂ Mn o M(i-1) is a subset of M(i) • Coherence Property o Copies of same information item at successive memory levels be consistent o Strategies to maintain Coherence: • 1) Write-Through (WT) • 2) Write-Back (WB) • Locality of Reference o Temporal: recently referenced items are likely to be referenced again in near future Dr. Vandana Jha, Department of CSE, KSSEM o Spatial: tendency of a process to access items whose addresses are 25

The Working Sets model ADVANCED PROCESSOR TECHNOLOGY Memory Hierarchy Technology Dr. Vandana Jha, Department of CSE, KSSEM 26

ADVANCED PROCESSOR TECHNOLOGY Memory Hierarchy Technology • Dr. Vandana Jha, Department of CSE, KSSEM 27

ADVANCED PROCESSOR TECHNOLOGY Memory Hierarchy Technology Virtual Memory Technology • Virtual memory is stored in the secondary storage device and helps to extend additional memory capacity. • Work with primary memory to load applications. • Reduces the cost of expanding the capacity of physical memory. • Implimentations differ from one OS to other. Dr. Vandana Jha, Department of CSE, KSSEM 28

Virtual Memory Technology • Each process address space is partitioned into parts and used for code, data and stack. • Parts are loaded into primary memory when needed and written back to secondary storage otherwise. • The logical address space is referred to as virtual memory. • Virtual memory is much larger than the physical memory. • Virtual memory uses: Virtual address and Physical address. • CPU translates Virtual address to Physical address. • Virtual memory system uses paging. Dr. Vandana Jha, Department of CSE, KSSEM 29

Locating an Object in a Cache 1. Search for matching tag o SRAM cache Object Name X = X? 0: 1: N-1: 2. Use indirection to look up actual object location • DRAM cache Lookup Table Object Name X D: J: X: Location 0 N-1 • • • 1 Dr. Vandana Jha, Department of CSE, KSSEM “Cache” Tag Data D 243 X 17 • • • J 105 “Cache” Data 0: 243 1: 17 • • • N-1: 105 30

A System with Physical Memory Only • Examples: o most Cray machines, early PCs, nearly all embedded systems, etc. Memory Physical Addresses 0: 1: CPU N-1: Addresses generated by the CPU point directly to bytes in physical memory Dr. Vandana Jha, Department of CSE, KSSEM 31

A System with Virtual Memory • Examples: o workstations, servers, modern PCs, etc. Virtual Addresses Page Table 0: 1: Memory 0: 1: Physical Addresses CPU P-1: N-1: Disk Address Translation: the hardware converts virtual addresses into physical addresses via an OS-managed lookup table (page table) Dr. Vandana Jha, Department of CSE, KSSEM 32

Page Faults (Similar to “Cache Misses”) • What if an object is on disk rather than in memory? o Page table entry indicates that the virtual address is not in memory o An OS exception handler is invoked, moving data from disk into memory • current process suspends, others can resume • OS has full control over the placement. Before fault After fault Memory Virtual Addresses Page Table Physical Addresses CPU Disk Dr. Vandana Jha, Department of CSE, KSSEM Disk 33

Servicing a Page Fault • Processor Signals Controller o Read block of length P starting at disk address X and store starting at memory address Y • Read Occurs o Direct Memory Access (DMA) o Under control of I/O controller • I/O Controller Signals Completion o Interrupt processor Dr. Vandana Jha, Department of CSE, KSSEM o OS resumes suspended (1) Initiate Block Read Processor Reg (3) Read Done Cache Memory-I/O bus (2) DMA Transfer Memory I/O controller disk Disk 34

Solution: Separate Virtual Addr. Spaces o Virtual and physical address spaces divided into equal-sized blocks • blocks are called “pages” (both virtual and physical) o Each process has its own virtual address space • operating system controls how virtual pages are assigned to physical memory Virtual Address Space for Process 1: 0 VP 1 VP 2 Address Translation 0 PP 2 . . . N-1 PP 7 Virtual Address Space for Process 2: 0 VP 1 VP 2 Dr. Vandana Jha, Department of CSE, KSSEM (e. g. , read/only library code) PP 10 . . . N-1 Physical Address Space (DRAM) M-1 35

Protection • Page table entry contains access rights information o hardware enforces this protection (trap into OS if violation occurs) Page Tables Read? Write? VP 0: Yes No Process i: VP 1: Yes VP 2: No • • • Physical Addr PP 9 Yes PP 4 No XXXXXXX • • • Read? Write? VP 0: Yes Process j: Memory • • • Physical Addr PP 6 VP 1: Yes No PP 9 No No XXXXXXX VP 2: • • • Dr. Vandana Jha, Department of CSE, KSSEM 0: 1: N-1: • • • 36

Virtual Memory Address Translation V = {0, 1, . . . , N– 1} virtual address space. N > M P = {0, 1, . . . , M– 1} physical address space MAP: V P U { } address mapping function MAP(a) = a' if data at virtual address a is present at physical address a' in P = if data at virtual address a is not present in P page fault handler Processor a virtual address Hardware Addr Trans Mechanism a' Main Memory part of the physical address on-chip memory mgmt unit (MMU) Dr. Vandana Jha, Department of CSE, KSSEM Secondary memory OS performs this transfer (only if miss) 37

Virtual Memory Address Translation • Parameters o P = 2 p = page size (bytes). o N = 2 n = Virtual address limit o M = 2 m = Physical address limit n– 1 virtual page number p p– 1 page offset 0 virtual address translation m– 1 p p– 1 physical page number page offset 0 physical address Notice that the page offset bits don't change as a result of translation Dr. Vandana Jha, Department of CSE, KSSEM 38

Page Tables Virtual Page Number Memory resident page table Valid (physical page or disk address) 1 1 0 1 0 1 Dr. Vandana Jha, Department of CSE, KSSEM Physical Memory Disk Storage (swap file or regular file system file) 39

Integrating VM and Cache VA CPU Translation miss PA data Cache Main Memory hit • Most Caches “Physically Addressed” o o Accessed by physical addresses Allows multiple processes to have blocks in cache at same time Allows multiple processes to share pages Cache doesn’t need to be concerned with protection issues • Access rights checked as part of address translation Dr. Vandana Jha, Department of CSE, KSSEM 40

Speeding up Translation with a TLB • “Translation Lookaside Buffer” (TLB) o Small hardware cache in MMU o Maps virtual page numbers to physical page numbers o Contains complete page table entries for small number of pages VA CPU TLB Lookup miss hit PA miss Cache Main Memory hit Translation data Dr. Vandana Jha, Department of CSE, KSSEM 41

References • Text Books: Kai Hwang and Naresh Jotwani, Advanced Computer Architecture (SIE): Parallelism, Scalability, Programmability, Mc. Graw Hill Education 3/e. 2015 • Reference Books: John L. Hennessy and David A. Patterson, Computer Architecture: A quantitative approach, 5 th edition, Morgan Kaufmann Elseveir, 2013 • Class Notes • Question Bank • NPTEL or other Videos: • https: //nptel. ac. in/courses/106102062/ Dr. Vandana Jha, Department of CSE, KSSEM 42