The Point of Views Multidimensional Views for Data

The Point of Views Multidimensional Views for Data Locality in HPC Cpp. Con 2017 Tobias Fuchs, LMU t. fuchs@mnm-team. org @fuchsto dash-project. org 1

http: //ibm-1401. info/Toronto-King. St-Datacenter-1. jpg Supercomputing in a Nutshell Back in the days when cores got faster, supercomputing was a gentlemen sport. 2

Supercomputing in a Nutshell Back in the days when cores got faster, supercomputing was a gentlemen sport. 3

How HPC used to be fun: “Increase Clock Frequency!” RAM The Point of Views – Multidimensional Index Sets 4

How HPC became complicated: “Add more cores!” RAM The Point of Views – Multidimensional Index Sets 5

How HPC became complex: “Add more sockets!” The Point of Views – Multidimensional Index Sets 6

How HPC stays difficult: “Oh, look, a PCIE slot” The Point of Views – Multidimensional Index Sets 7

How HPC stays difficult: “So we heard you like architectures” L 3 IC ASIC, DSP, … The Point of Views – Multidimensional Index Sets 8

Like moving Ikea furniture with your closest 4096 friends Host Interconnect GPU The Point of Views – Multidimensional Index Sets 9

Like moving Ikea furniture with your closest 4096 friends have screwdrivers, can lift heavy stuff expensive truck Host Interconnect narrow hallway mom‘s good china GPU have electric drills, hate going outside strong guys, only speak finno-ugrian esperanto The Point of Views – Multidimensional Index Sets 10

Example: Multigrid Subdivision in 3 D Heat Equation TU Dresden Andreas Knüpfer Denis Hünich github. com/dash-project/dash-apps - Constant temperature forced on two outside planes of a cube Simulation finds steady state (heat dissipation in volume does not change any more) The Point of Views – Multidimensional Index Sets 11

Halo Exchange (2 D) 0. 1 0. 6 0. 1 The Point of Views – Multidimensional Index Sets 12

Halo Exchange (2 D) The Point of Views – Multidimensional Index Sets 13

Halo Exchange (2 D) The Point of Views – Multidimensional Index Sets 14

Halo Exchange (2 D) The Point of Views – Multidimensional Index Sets 15

Halo Exchange (2 D) The Point of Views – Multidimensional Index Sets 16

Halo Exchange (2 D) The Point of Views – Multidimensional Index Sets 17

Halo Exchange (2 D) Process A Process B The Point of Views – Multidimensional Index Sets 18

Halo Exchange (2 D) Process A Process B The Point of Views – Multidimensional Index Sets 19

Halo Exchange (2 D) Process A Process B The Point of Views – Multidimensional Index Sets 20

Halo Exchange (2 D) Process A Process B The Point of Views – Multidimensional Index Sets 21

Halo Exchange (2 D) Process A Process B The Point of Views – Multidimensional Index Sets 22

Halo Exchange (2 D) Process A Process B The Point of Views – Multidimensional Index Sets 23

Halo Exchange (2 D) Process A Process B The Point of Views – Multidimensional Index Sets 24

Classic Problems in HPC Data Access Reduce number of remote accesses - copy remote data in contiguous chunks - pack strided data into contiguous send/receive buffers - replicate data ranges … The Point of Views – Multidimensional Index Sets 25

Classic Problems in HPC Data Access Reduce number of remote accesses - copy remote data in contiguous chunks - pack strided data into contiguous send/receive buffers - replicate data ranges … Reduce communication volume - improve surface to volume ratio … The Point of Views – Multidimensional Index Sets 26

Classic Problems in HPC Data Access Reduce number of remote accesses - copy remote data in contiguous chunks - pack strided data into contiguous send/receive buffers - replicate data ranges … Reduce communication volume - improve surface to volume ratio … Reduce access distance ( locality) - pin cooperating processes close to each other in hardware hierarchy (virtual topologies) - subdivide data domain hierarchically to match hardware topology … The Point of Views – Multidimensional Index Sets 27

Partitioned Global Address Space dash: : Array<int> arr(40); The Point of Views – Multidimensional Index Sets 28

Partitioned Global Address Space dash: : Array<int> arr(40); Segment in Global Address Space gpointer: { segment, unit, local offset } 29 The Point of Views – Multidimensional Index Sets

Containers in Partitioned Global Address Space dash: : Array<int> arr( size, dash: : BLOCKED); dash: : Array<int> arr(40); local access remote access if (dash: : myid() == 2) int x = arr[27] + arr[35]; The Point of Views – Multidimensional Index Sets 30

Containers in Partitioned Global Address Space dash: : Array<int> arr( size, dash: : BLOCKED); auto int dash: : Array<int> arr(40); l_arr = local(arr); l_size = l_arr. size(); * l_bg = l_arr. begin(); int x = arr. local[2] + arr. local[8]; explicit local access The Point of Views – Multidimensional Index Sets 31

Algorithms on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( arr. begin(), arr. end(), unary_op); Same code executed by every unit, classic SIMD auto l_end = std: : for_each( arr. local(). begin(), arr. local(). end(), unary_op); arr. barrier(); // communication: O(1) return arr. end(); The Point of Views – Multidimensional Index Sets 32

Algorithms on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( dash: : launch: : async*, arr. begin(), arr. end(), unary_op); * work in progress, will be as close to C++ STL concepts as possible auto l_end = std: : for_each( arr. local(). begin(), arr. local(). end(), unary_op); arr. barrier(); // communication: O(0) return arr. end(); The Point of Views – Multidimensional Index Sets 33

Algorithms on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( arr. begin() + 12, arr. begin() + 28, unary_op); The Point of Views – Multidimensional Index Sets 34

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( arr. begin() + 12, arr. begin() + 28, unary_op); Using views: auto l_range = arr | sub(12, 28) | local(); // view at local range (2… 10)(0… 8)(0… 0) std: : for_each(l_range. begin(), l_range. end(), unary_op); The Point of Views – Multidimensional Index Sets 35

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( arr. begin() + 12, arr. begin() + 28, unary_op); “But but there already is range-v 3 and it rocks? “ Using views: auto l_range = arr | sub(12, 28) | local(); // view at local range (2… 10)(0… 8)(0… 0) std: : for_each(l_range. begin(), l_range. end(), unary_op); The Point of Views – Multidimensional Index Sets 36

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( Yes! arr. begin() + 12, arr. begin() + 28, unary_op); github. com/ericniebler/range-v 3 … at every unit: auto l_range = arr | sub(12, 28) | local(); // view at local range (2… 10)(0… 8)(0… 0) std: : for_each(l_range. begin(), l_range. end(), unary_op); The Point of Views – Multidimensional Index Sets 37

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( Yes! arr. begin() + 12, arr. begin() + 28, unary_op); github. com/ericniebler/range-v 3 … at every unit: auto l_range = arr | sub(12, 28) | local(); // view at local range (2… 10)(0… 8)(0… 0) std: : for_each(l_range. begin(), l_range. end(), unary_op); The Point of Views – Multidimensional Index Sets 38

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); auto end = dash: : for_each( arr. begin() + 12, arr. begin() + 28, unary_op); But we have “domainspecific” needs Using views: auto l_range = arr | sub(12, 28) | local(); // view at local range (2… 10)(0… 8)(0… 0) std: : for_each(l_range. begin(), l_range. end(), unary_op); The Point of Views – Multidimensional Index Sets 39

Mental Model: Set Mappings Domain Image The Point of Views – Multidimensional Index Sets 40

Mental Model: Set Mappings Domain Image The Point of Views – Multidimensional Index Sets 41

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); The Point of Views – Multidimensional Index Sets 42

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); arr | sub(12, 28) The Point of Views – Multidimensional Index Sets 43

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); arr | sub(12, 28) | local() The Point of Views – Multidimensional Index Sets 44

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); arr | sub(12, 28) arr | | | // -> sub(12, 28) local() index() { 0 … 8 } The Point of Views – Multidimensional Index Sets 45

Views on PGAS Containers dash: : Array<int> arr( size, dash: : BLOCKED); arr | sub(12, 28) arr | | // -> sub(12, 28) local() index() global() { 20 … 28 } The Point of Views – Multidimensional Index Sets 46

Copy Algorithm Example dash: : Array<int> src(size); dash: : Array<int> dst(src. size()); auto cp_end = dash: : copy( src. begin() + sa, src. begin() + sb, dst. begin() + da); The Point of Views – Multidimensional Index Sets 47

Copy Algorithm Example dash: : Array<int> src(size); dash: : Array<int> dst(src. size()); auto cp_end = dash: : copy( src. begin() + sa, src. begin() + sb, dst. begin() + da); element-wise copying is FORBIDDEN – INTERDIT - DAS VERBOTEN The Point of Views – Multidimensional Index Sets 48

Copy Algorithm Example dash: : Array<int> src(size); dash: : Array<int> dst(src. size()); auto cp_end = dash: : copy( src. begin() + sa, src. begin() + sb, dst. begin() + da); The Point of Views – Multidimensional Index Sets 49

Copy Algorithm Example dash: : Array<int> src(size); dash: : Array<int> dst(src. size()); auto cp_end = dash: : copy( src. begin() + sa, src. begin() + sb, dst. begin() + da); The Point of Views – Multidimensional Index Sets 50

Copy Algorithm Example dash: : Array<int> src(size); dash: : Array<int> dst(src. size()); auto cp_end = dash: : copy( src. begin() + sa, src. begin() + sb, dst. begin() + da); The Point of Views – Multidimensional Index Sets 51

Copy Algorithm Example auto src_blks = src | sub(sa, sb) | local() | blocks(); if (!src_lbs) return dst. begin() + db; auto dst_blks = dst | sub(da, db) | blocks(); // { { da … 20 }, { 20 … db } } for (auto & bk : bk_dst) { … } The Point of Views – Multidimensional Index Sets 52

The Point of Views Takeaway message: In most cases, we don’t access every element in a view, we only care about their emergent properties like boundaries of contiguous blocks. Once we have a contiguous range, we pass its native local address range (T*) to the transport API (like MPI). Lazy computation like range-v 3 but focus on domain mappings, not generating sequential ranges. The Point of Views – Multidimensional Index Sets 53

Local Regions of Global Subrange one of 4 specializations (and counting) mpirun -n 3 ex. 02. array-views. mpi The Point of Views – Multidimensional Index Sets 54

Local Regions of Global Subrange auto sub_loc_idx = range | sub(a, b) | local() | index(); vs. works for anything that can represent an n-dim rectangle (n=1 range) mpirun -n 3 ex. 02. array-views. mpi The Point of Views – Multidimensional Index Sets 55

HPC Developers be like. . . (Dramatization, but not really) Naah, I don’t want to maintain illegible ivory tower template voodoo in my code! Manual index calculations are fine and dandy, I just copy them from the original FORTRAN 77 code. And, you must know, every abstraction has its performance overhead. mpirun -n 4 bench. 05. array-range. mpi I saw some C++ in 1998 and it looked like an AT&T port of Plankalkül. The Point of Views – Multidimensional Index Sets 56

HPC Developers be like. . . (Dramatization, but not really) *Humpph* auto sub_loc_idx = range | sub(a, b) | local() | index(); The Point of Views – Multidimensional Index Sets 57

View Operations begin() end() size() rank() offsets() extents() domain() origin() index() local() global() pattern() blocks() chunks() join() stride(i 0…id) sub<d>(b, e) expand<d>(ob, oe) shift<d>(o) halo() border() is_local_at(u) is_contiguous() is_global() is_local() view_traits<V> iterator value_type. . . is_local_view is_global_view is_origin rank The Point of Views – Multidimensional Index Sets 58

View Operations begin() end() size() rank() offsets() extents() domain() origin() index() local() global() pattern() blocks() chunks() join() stride(i 0…id) sub<d>(b, e) expand<d>(ob, oe) shift<d>(o) halo() border() is_local_at(u) is_contiguous() is_global() is_local() view_traits<V> iterator value_type. . . is_local_view is_global_view is_origin rank The Point of Views – Multidimensional Index Sets 59

Multidimensional Distribution Patterns Pattern concept specifies data distribution as traits in three categories: Global Canonical Domain The Point of Views – Multidimensional Index Sets 60

Multidimensional Distribution Patterns Pattern concept specifies data distribution as traits in three categories: Global Canonical Domain 1. Partitioning The Point of Views – Multidimensional Index Sets 61

Multidimensional Distribution Patterns Pattern concept specifies data distribution as traits in three categories: Global Canonical Domain 1. Partitioning 2. Process Mapping The Point of Views – Multidimensional Index Sets 62

Multidimensional Distribution Patterns Pattern concept specifies data distribution as traits in three categories: Global Canonical Domain 1. Partitioning 2. Process Mapping 3. Memory Layout The Point of Views – Multidimensional Index Sets 63

Multidimensional Distribution Patterns Pattern concept specifies data distribution as traits in three categories: Global Canonical Domain 1. Partitioning 2. Process Mapping 3. Memory Layout 0 1 2 3 4 5 6 … The Point of Views – Multidimensional Index Sets 64

Multidimensional Distribution Patterns of static data structures (Array, NArray, Co. Array, …) are static mappings, constructor is final specification (soon: at compile time) Global Canonical Domain 1. Partitioning 2. Process Mapping 3. Memory Layout 0 1 2 3 4 5 6 … The Point of Views – Multidimensional Index Sets 65

Multidimensional Distribution Patterns Distribution semantics can fry your frontal lobe. Some implications of just three mapping traits: Mapping: neighbor, unbalanced neighbor, balanced diagonal, balanced … and it gets even worse with fancy virtual process topologies The Point of Views – Multidimensional Index Sets 66

Multidimensional Distribution Patterns - We add new distribution schemes all the time - Cumbersome for users to keep track of pattern types’ semantics We provide: - Deduction of distribution type from set of properties typedef typename make_pattern< partitioning_properties< partitioning: : balanced_tag >, mapping_properties< mapping: : diagonal_tag > >: : type my_pattern_t; Matrix<Value. T, 2, my_pattern_t> m(…); mpirun -n 1. /bin/ex. 06. make-pattern. mpi The Point of Views – Multidimensional Index Sets 67

Multidimensional Distribution Patterns - We add new distribution schemes all the time - Cumbersome for users to keep track of pattern types’ semantics We provide: - Deduction of distribution type from set of properties - Visualizer for user-specified distributions mpirun -n 1. /bin/ex. 06. pattern-block-visualizer. mpi -s shift -n 9 9 -t 3 3 -u 1 3 The Point of Views – Multidimensional Index Sets 68

Multidimensional Views / Ranges / Index Sets* HPC special needs: dim 0 dash: : Matrix<int, 2> mat(8, 8, … ); dim 1 * we use the term index set to differentiate from index sequences The Point of Views – Multidimensional Index Sets 69

Multidimensional Views / Ranges / Index Sets* HPC special needs: dim 0 dash: : Matrix<int, 2> mat(8, 8, … ); dim 1 * we use the term index set to differentiate from index sequences The Point of Views – Multidimensional Index Sets 70

Multidimensional Views / Ranges / Index Sets HPC special needs: dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 The Point of Views – Multidimensional Index Sets 71

Multidimensional Views / Ranges / Index Sets HPC special needs: dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 auto l_mat = mat | local(); Two iteration orders: - canonical order vs. - storage order The Point of Views – Multidimensional Index Sets 72

Multidimensional Views / Ranges / Index Sets dim 1 dim 0 canonical order storage at unit 0 The Point of Views – Multidimensional Index Sets 73

Multidimensional Views / Ranges / Index Sets dim 1 dim 0 canonical order storage at unit 1 The Point of Views – Multidimensional Index Sets 74

Multidimensional Views / Ranges / Index Sets dim 1 dim 0 canonical order storage at unit 2 The Point of Views – Multidimensional Index Sets 75

Multidimensional Views / Ranges / Index Sets It is plausible dim 1 that dim 1 dim 0 iteration order = storage order would help locality. But it is not portable/generic. Also: pointers vs. iterators canonical order storage at unit 2 The Point of Views – Multidimensional Index Sets 76

Multidimensional Views / Ranges / Index Sets The role of STL concepts in DASH dim 0 What result do you expect? dim 1 std: : vector<int> row_copy(24); std: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); The Point of Views – Multidimensional Index Sets 77

Multidimensional Views / Ranges / Index Sets The role of STL concepts in DASH dim 0 What result do you expect? dim 1 std: : vector<int> row_copy(24); std: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); “Multidimensional arrangement only exists in your head, memory is linear. ” The Point of Views – Multidimensional Index Sets 78

Multidimensional Views / Ranges / Index Sets The role of STL concepts in DASH dim 0 What result do you expect now? dim 1 std: : vector<int> row_copy(24); std: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); The Point of Views – Multidimensional Index Sets 79

Multidimensional Views / Ranges / Index Sets The role of STL concepts in DASH dim 0 What result do you expect now? dim 1 std: : vector<int> row_copy(24); std: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); “Algorithms are not concerned with memory layout, silly. Just iterators. ” The Point of Views – Multidimensional Index Sets 80

Multidimensional Views / Ranges / Index Sets The role of STL concepts in DASH dim 0 What result do you expect now? dim 1 std: : vector<int> row_copy(24); std: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); “Algorithms are not concerned with memory layout, silly. Just iterators. ” The Point of Views – Multidimensional Index Sets 81

Multidimensional Views / Ranges / Index Sets The role of STL concepts in DASH dim 0 What result do you expect now? dim 1 std: : vector<int> row_copy(24); std: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); “Algorithms are not concerned with memory layout, silly. Just iterators. ” The Point of Views – Multidimensional Index Sets 82

Multidimensional Views / Ranges / Index Sets The role of STL concepts in DASH dim 0 std: : vector<int> row_scp(24); // takes ages, but works std: : copy( mat. begin() + 8, mat. begin() + 32, row_scp. begin()); dim 1 std: : vector<int> row_dcp(24); // same result dash: : copy(mat. begin() + 8, mat. begin() + 32, row_dcp. begin()); The Point of Views – Multidimensional Index Sets 83

Multidimensional Views / Ranges / Index Sets dim 1 dim 0 Algorithm implementations adapt to exploit decomposition properties for performance but canonical semantics are consistent. std: : vector<int> row_copy(24); dash: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); The Point of Views – Multidimensional Index Sets 84

Multidimensional Views / Ranges / Index Sets dim 1 dim 0 Algorithm implementations adapt to exploit decomposition properties for performance but canonical semantics are consistent. std: : vector<int> row_copy(24); dash: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); The Point of Views – Multidimensional Index Sets 85

Multidimensional Views / Ranges / Index Sets dim 1 dim 0 Algorithm implementations adapt to exploit decomposition properties for performance but canonical semantics are consistent. std: : vector<int> row_copy(24); dash: : copy(mat. begin() + 8, mat. begin() + 32, row_copy. begin()); The Point of Views – Multidimensional Index Sets 86

Multidimensional Views / Ranges / Index Sets View Algebra dim 0 dash: : Matrix<int, 2> mat(8, 8, … ); dim 1 auto mat_rect = mat | sub<0>(2, 4) | sub<1>(2, 7); The Point of Views – Multidimensional Index Sets 87

Multidimensional Views / Ranges / Index Sets View Algebra dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 auto mat_rect = mat | sub<0>(2, 4) | sub<1>(2, 7) | local(); mat_rect | | // result, unit 0: { unit 1: { unit 2: { index() join(); depending on unit: 8, 9, 10, 11 } 6, 7, 8, 9 } 4, 8 } mpirun -n 2 ex. 02. matrix-views. mpi The Point of Views – Multidimensional Index Sets 88

mpirun -n 4 ex. 02. matrix-local-views. mpi 89

Multidimensional Views / Ranges / Index Sets View Algebra dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 auto mat_rect = mat | block({1, 1}); mpirun -n 2 ex. 02. matrix-halo-views. mpi The Point of Views – Multidimensional Index Sets 90

Multidimensional Views / Ranges / Index Sets View Algebra dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 auto mat_rect = mat | block({1, 1}) | expand({-1, 0}, {-1, 1}); mpirun -n 2 ex. 02. matrix-halo-views. mpi The Point of Views – Multidimensional Index Sets 91

mpirun -n 3 ex. 02. matrix-halo-views. mpi 92

mpirun -n 3 ex. 02. matrix-halo-views. mpi 93

Multidimensional Views / Ranges / Index Sets View Algebra dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 auto mat_rect = mat | block({1, 1}) | expand({-1, 0}, {-1, 1}); mpirun -n 2 ex. 02. matrix-halo-views. mpi The Point of Views – Multidimensional Index Sets 94

Multidimensional Views / Ranges / Index Sets View Algebra dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 auto mat_rect = mat | block({1, 1}) | expand({-1, 0}, {-1, 1}) | shift<1>(2); mpirun -n 2 ex. 02. matrix-halo-views. mpi The Point of Views – Multidimensional Index Sets 95

Multidimensional Views / Ranges / Index Sets View Algebra dim 0 dash: : Matrix<int, 2> mat(8, 8, { TILED(3), TILED(3) }); dim 1 auto mat_rect = mat | block({1, 1}) | expand({-1, 0}, {-1, 1}) | shift<1>(2) | intersect( block(5)); mpirun -n 4 ex. 02. matrix-sub-views. mpi The Point of Views – Multidimensional Index Sets 96

mpirun -n 4 ex. 02. matrix-sub-views. mpi The Point of Views – Multidimensional Index Sets 97

mpirun -n 4 ex. 02. matrix-sub-views. mpi The Point of Views – Multidimensional Index Sets 98

mpirun -n 4 ex. 02. matrix-sub-views. mpi 99

mpirun -n 4 ex. 02. matrix-sub-views. mpi 100

dash-project. org github. com/dash-project. slack. com doc. dash-project. org/papers doc. dash-project. org/slides doc. dash-project. org/files/wallpapers 101