Parallel Computing Basic Concepts Computational Models Synchronous vs

Parallel Computing • Basic Concepts • Computational Models • Synchronous vs. Asynchronous • The Flynn Taxonomy • Shared versus Distributed Memory • Interconnection networks • Bus structures • Static • Dynamic • Performance Issues • Amdahl versus Gustafson-Barsis

Parallel Computing cont. • Parallel Algorithms • Languages for Parallel Computing • PVM and MPI • Memory Issues • To be treated with memory in general

Basic Concepts • Introduction • • Why? … the von Neumann Bottleneck Problems … the Grand Challenges The Gordon Bell Prize Common parallelism • buses • piplining • Computational Models • Synchronous vs Asychronous • The Flynn Taxonomy INSTRUCTIONS DATA SISD SIMD MISD MIMD

Flynn Taxonomy • SISD • superscalar architectures • VLIW • Vector processors • SIMD • fine grained parallelism • generally many processors • MISD • not used • MIMD • coarse grained parallelism • few processors Super-Flynn • SPMD • single program … multiple data • very common

Other Catagorizations Memory • Shared versus Distributed • tightly coupled versus loosely coupled • multiprocessor versus multicomputer Interconnects • Static versus Dynamic • Hardwired versus switched

Shared Memory • Advantages • Price/performance … use “off the shelf” components • Simplicity … reliability • Scalability • Disadvantages • Device contention • Local cache coherency • Topologies • bus contention processes • 1 -D multiple buses • crossbar switch • multiport memory • 2 -D multiple buses • hierarchical bus architecture • NUMA … non-uniform shared memory … KSR 1

Static Interconnects • Goals • Fewest interconnects • Fewest hops • 1 -D Linear • 2 -D Ring • Grid / Mesh • Toroid • Fully Connected • Tree, Quadtree • Fat Tree • Hypercube Dynamic Interconnects • Crossbar • Delta • CLOS

Shared Memory Machines • Sequent Symmetry … bus • FX/8 … bus • Cray Y-MP … multiport memory Distributed Memory Machines • Intel Paragon … 2 -D grid • Wavetracer … 3 -D grid • Mas. Par MP-2 2 -D torus and 3 -stage CLOS • n. CUBE … hypercube • i. PSC/2 … hypercube • CM - 2 … 2 -D grid and 12 -D hypercube • CM-5 … Fat Tree • Kendall Square Research KSR 1 … rings • Alliant Campus … crossbars (multi) • Paradigm … bus(es) and interconnect