Parallel primitives Map Same operation on the input

Parallel primitives

Map �Same operation on the input elements �Parallel execution of a serial iteration Input Elemental Function Output Introduction to Parallel Computing, University of Oregon, IPCC

MAP �SAXPY (Scaled Vector Addition) y = ax + y BLAS (Basic Linear Algebra Subprograms) 0 1 2 3 4 5 6 7 8 9 10 11 4 4 4 2 4 2 1 8 3 9 5 5 1 2 1 + y 3 7 0 1 4 0 0 4 5 3 1 0 y 11 23 8 5 36 12 36 49 50 7 9 4 a * x Introduction to Parallel Computing, University of Oregon, IPCC

Reduce �A combination of input elements Associative binary operations Min, max, add, sub Introduction to Parallel Computing, University of Oregon, IPCC

Reduce �Partitioned reduction Introduction to Parallel Computing, University of Oregon, IPCC

Scan �All possible partial reduction of the input elements Each element of the output is the reduction of the input elements till that output element Exclusive or inclusive Introduction to Parallel Computing, University of Oregon, IPCC

Scan �Naive parallelization A B C D E A A+B B+C C+D D+E A A+B+C+D B+C+D+E A A+B+C+D+E

Scan �Work effective implementation Blelloch 1990, balanced binary tree Two phases ▪ Up sweep: ▪ Calculating sub sums from the leaf nodes to the root ▪ The root contains the whole sum ▪ Down sweep: ▪ Cumulative sub sums from the root the leaf nodes ▪ In case of exclusive scan the root element is replaced by zero

Scan Up sweep Down sweep Introduction to Parallel Computing, University of Oregon, IPCC

Gather and scatter �Gather Which element of the input shall be placed to the output �Scatter Where to put the input elements in the output

Compact �Conditional selection �Selecting elements that meet a criterion Map, scan, map Introduction to Parallel Computing, University of Oregon, IPCC