Hierarchically Tiled Arrays Presented by Kenneth Detweiler Overview

Hierarchically Tiled Arrays Presented by, Kenneth Detweiler

Overview v Design and Creation v Background v Parallel Operators v Tiling v Locality v HTA v v Illustrated Creation Data Layout Accessing v Implementation v Sparse Matrix v Cannon Algorithm v Estimate PI using Monte Carlo method v MATLAB v Internal Structure v What it does v C++ Implementation v v Logical Index Space HTA Class Machine Mapping Operator Framework v C++ Optimizations v v v Automatic Memory Management Template Class Specialized Methods Inlined Hot Methods Lazy Evaluation Relaxation of serial evaluation v Current State of HTA v My Thoughts

Design and Creation • Developed in 2006 by: ▫ University of Coruna ▫ University of Illinois ▫ IBM • IBM: Yorktown Heights, NY • UC: Coruna, Spain • UIC: Chicago, Illinois

Background • Data type based on higher level data parallel operators ▫ Hierarchical Tiling • HTA creates tiles a part of programming language ▫ Tiling helps control locality and Data Distribution ▫ Referenced explicitly by the compiler ▫ Operators extend to function on tiles • C++ Library Implementation ▫ Distributed Memory using MPI ▫ Shared Memory using TBI

Parallel Operators • In order to give benefits of supercomputer power the developer resorts to low level parallel constructs. ▫ This is both time consuming and an easily error prone process • The solution to this Parallel Operators, which exists simultaneously on all processors involved in distributed contribution. ▫ Acts as a single entity capable of processing shared data in parallel.

Hierarchical Tiling • Arrays that are partitioned into tiles • Exploits parallelism and locality of all levels of memory hierarchy. • Can be represented iteratively or recursively. • Implemented during program design • Cut costs in parallel programs by organizing computations between tiles

Locality • Known as the locality of reference or principle of locality • Describes the same value or related storage locations being frequently accessed. • Two major types: ▫ Temporal: �If a particular memory location is referenced then it is likely that same location will be referenced again in the future. ▫ Spatial: �If a particular memory location is referenced then it is likely that the nearby memory locations will also be referenced

HTA, Illustrated

HTA, Creation • Need: ▫ Source Array ▫ Series of Delimiters • Function: ▫ hta(MATRIX{[delim 1], [delim 2], …}, [processor mesh]) • Example: ▫ h = hta(a, {[1, 4]. , [1, 3]}, [2, 2]);

HTA, Data Layout • Inner most tiles are the leaf tiles. • Leaf tiles contain: ▫ ROW ▫ COLUMN ▫ TITLE: stores the elements of tile in continuous memory locations • Memory mapping of HTA is determined by: ▫ How tiles are allocated ▫ Memory layout of tiles

HTA, Accessing • Tiles: ▫ Calling C{2, 2} is the lower right quadrant • Elements: ▫ Calling C(2, 2)

Parallel Programming using HTA Sparse Matrix a = hta(MX, {dist}, [P 1]); b = hta(P, 1, [P 1]); b{: } = V; r = a * b; Communication between client and each server Multiples a sparse matrix mx by a dense vector V using P processor. Distribute MX in chunks of rows into a HTA by calling the constructor P servers handling the HTA are organized into a single column Dist argument is used to distribute the array MX such that it results in a uniform computation across the servers

Parallel Programming using HTA Cannon Algorithm for i = 1: n c = c + a * b; a = circshift(a, [0, -1]); b = circshift(b, [-1, 0]); end Requires communication between client and servers, but also communication between the servers Each iteration of the loop each server executes a matrix multiplication of tile a and b that reside on the server Result of multiplication is stored in a local HTA c Tiles a are shifted along the first dimension, and tiles b are shifted along second dimension. Thus tiles a are sent to the left processor in mesh and tiles b to the right processor in mesh. The left processor sends its tiles of a to the right most processor in its row and the bottom most processor transfers its tiles of b to the top most processor in its column The end result is that HTA c = a*b

Parallel Programming using HTA Estimate pi using Monte Carlo method input = hta(P, 1, [P 1]); input{: } = e. P; output = par. HTAFunc(@estimate. Pi, input); my. Pi = mean(output(: )); function r = estimate. Pi(n) x = randx(1, n); y = randx(1, n); pos = x. * x + y. * y; r = sum(pos < 1) / n * 4; A distributed HTA input with one tile per processor is built Tiles are filled with e. P, the number of experiments to run per processor Experiments are made on each processor by the function estimate. Pi() The result of the parallel execution of the function is a distributed HTA output that has the same mapping as input and keeps a single tile per processor with the local estimation of pi.

Implementation • MATLAB (matrix laboratory) ▫ A numerical computing environment and fourth generation programming language ▫ Started as a wrapper on Fortran libraries

MATLAB – Internal Structure • MATLAB is used as a client where code is executed • On the server where its used as a computational engine for the distributed operations on the HTA’s.

MATLAB – Internal Structure (cont) • All communications are done through the MPI ▫ Lower layers of the HTA toolbox take care of communications requirements. ▫ Higher layers implement the syntax expected by the MATLAB users • HTA programs have a single thread that is interpreted and executed by the MATLAB client • HTA’s are just objects within the environment of the interpreter

MATLAB – Internal Structure (cont) • When HTA is local it is not distributed on the array servers ▫ Client HTA keeps both the structure and the content of the HTA • When HTA is distributed ▫ Client holds the structure of the HTA at all its levels ▫ Keeps all information of the mapping of tiles on the top level of the mesh servers • Client is always able to regardless of local or distributed to: ▫ Test the legality of the operation ▫ Calculate the structure and mapping of the output HTA ▫ Send messages that encode the command its arguments to the servers

MATLAB • Is a linear algebra language with a large base of users who write scientific code ▫ HTA allows users to harness powers of cluster of work stations instead of a single machine • Is polymorphic ▫ Allowing HTA s to substitute regular arrays almost without changing the rest of the code, thereby adding parallelism painlessly. • Is designed to be extensible ▫ Third party developers can provide so call toolboxes of functions for specialized purposes. • Provides a native method called Mex ▫ Allows functions to implemented in languages like C or Python

C++ Implementations • HTAs can be implemented in C++ by adding the htalib library ▫ HTAs are represented as composite objects with methods to operate on both distributed and sequential HTAs ▫ Two communication layers are available MPI and UPC �Implementation follows SPMD execution model while the programming model is still single-threaded • Core Data Structures of htalib ▫ ▫ Logical index space HTA Class Machine Mapping Operator Framework

C++ Implementations Logical Index Space • Classes used to define index space and tiling of an HTA ▫ Tuple<N> //an n dimensional index value ▫ Triplet //a 1 D range with optional stride �(low: high: step) ▫ Region<N> //N-dimensional rectangular index space spanned by N triplets

C++ Implementations HTA Class • Defines an HTA with scalar elements of type T and N dimensions. • Data type implementation scalar access ▫ (operator[], tile access(operator()) • Built in array operations ▫ ▫ transpose permute dpermute reduce

C++ Implementation Machine Mapping • Specifies where the HTA is allocated in a distributed system • Memory layout of the scalar data array ▫ Captured by instances of class distribution that specifies home location of the scalar data for each of the tiles of an HTA • Memory Mapping ▫ Specify the layout �(row-major across tiles, row major per tile) ▫ Size and Stride of the flat array data underlining the HTA

C++ Implementation Operator Framework • Htalib provides a powerful operator framework following the design of the STL operator class ▫ Consists of routines that evaluate specific operators on HTAs and base classes �Serves as a foundation for user-defined operator

C++ Optimizations • Represented as composite objects ▫ Methods available for both distributed and local HTA’s ▫ Two implementations MPI and UPC • Performance Optimizations ▫ ▫ ▫ Automatic Memory Management Template Class Specialized Methods Inlined Hot Methods Lazy evaluation Relaxation of serial evaluation semantics

C++ Optimizations Automatic Memory Management • HTA’s are allocated through factory methods on the heap for automatic memory management ▫ Methods return handle which is assigned to a stack allocated variable ▫ All access occurs through this handle ▫ Once all handles to an HTA disappear from the stack the HTA and its related structures are automatically deleted from memory

C++ Optimizations Template Class • Used in htalib to handle data with different types and dimensions • Provides flexibility and opportunities for optimizations at compile time • Data type of HTA can be any type or user defined type

C++ Implementation Specialized Methods • Methods are optimized and whenever possible specialized for specific cases • IE: A specialized method that avoids multiplication by stride is invoked when the data being accessed is known to be stored consecutive locations.

C++ Implementation Inlined Hot Methods / Lazy Evaluation • Inlined Hot Methods ▫ Inlining is performed to methods that are used frequently ▫ The tile access functions and scalar functions are carefully inlined to reduce the overhead of function calls. • Lazy Evaluation ▫ In HTA Assignment when RHS has more than one variable htalib uses lazy evaluation to avoid or reduce the temporary variables generated from one or more binary operations ▫ Then if the LHS and RHS have no data dependencies then the assignment is directly evaluated to the RHS

C++ Examples Matrix Multiplication

C++ Implementation Relaxation of serial evaluation semantics • Htalib provides a mechanism to temporarily relax the serial evaluations ordering ▫ Helps the overlapping of different communications and of communications with computations htalib: : async(); B(1: n)[0] = B(0: n-1)[d]; B(0: n-1)[d+1] = B(1: n)[1]; htalib: : sync(); ▫ The above code shows the boundary exchange in the 1 D Jacobi ▫ There is no data dependence among the assignments, both statements can proceed concurrently ▫ This is achieved through the runtime calls to async and sync

C++ Examples Matrix Transposition

State • Currently HTA library support is being extended to support ▫ Sparse Data Partitioning ▫ Hierarchical Place Trees • Continually optimized to increase performance • Searching for new ways to implement into new languages to increase productivity

My Thoughts By increasing the productivity of parallel computing we can increase processor power across machines More powerful programming languages due to having powerful library tools HTA is something that I will keep in mind with my future programming projects