Short Introduction to Parallel Computing CS 6560 Operating

(Short) Introduction to Parallel Computing CS 6560: Operating Systems Design

Why Parallel Computing? Performance! Many applications require serious performance. Examples: Structural biology Chemical dynamics Pharmaceutical design Weather forecasting Human genome Ocean modeling 2

Processor Performance: Need Parallelism! 2 -3 GHz 3

Case Study 1: Simulating Ocean Currents Model as two-dimensional grids Discretize in space and time finer spatial and temporal resolution => greater accuracy Many different computations per time step set up and solve equations (a) Cross sections Concurrency across and within grid computations (b) Spatial discretization of a cross section 4

Case Study 2: Simulating Galaxy Evolution Simulate interactions of many stars evolving over time Computing forces is expensive O(n 2) brute force approach Hierarchical methods take advantage of force law: G 5 m 1 m 2 r 2

Case Study 2: Barnes-Hut Many time steps, plenty of concurrency across stars Locality Goal Particles close together in space should be on same processor Spatial Domain Quad-tree Difficulties: Non-uniform, dynamically changing 6

Case Study 3: Rendering by Ray Tracing Goal is to produce image from representation of real world 3 D Scene Shoot rays into scene through pixels in projection plane Projection Plane Result is color for pixel Dynamically generated ray Rays shot through pixels in projection plane are called primary rays Reflect and refract when they hit objects Ray from viewpoint to upper right corner pixel Recursive process generates ray tree per primary ray Tradeoffs between execution time and image quality 7 Viewpoint

Partitioning Need dynamic assignment Use contiguous blocks to exploit spatial coherence among neighboring rays, plus tiles for task stealing A tile, the unit of decomposition and stealing A block, the unit of assignment 8

Sample Speedups Speedup = (best) time on 1 processor / time on multiple processors Speedups on NUMA multiprocessor 9

Ideal/Linear Speedup? Amdahl’s Law If a fraction s of a computation is not parallelizable, then the best achievable speedup is 10

Pictorial Depiction of Amdahl’s Law 1 Sequential work Parallelizable work p 1 Time 11

But Goal is not just Performance At some point, we’re willing to trade some performance for: Ease of programming High portability Low cost Ease of programming & high portability Parallel programming for the masses Leverage new or faster hardware asap Low cost High-end parallel machines are expensive resources 12

Parallel Applications Scientific computing not the only class of parallel applications Examples of non-scientific parallel applications: Data mining Real-time rendering Distributed servers Today, programmers are encouraged to find parallelism in all sorts of software 13