A Polymorphic Type System for Bulk Synchronous Parallel

A Polymorphic Type System for Bulk Synchronous Parallel ML Frédéric Gava and Frédéric Loulergue Laboratory of Algorithms, Complexity and Logic Paris, France F. Loulergue Pa. CT 2003

Overview n n n Introduction Pure functional bulk synchronous parallel programming Problems with nesting Overview of the system Conclusion and future work F. Loulergue Pa. CT 2003 2

Introduction F. Loulergue Pa. CT 2003

Bulk Synchronous Parallelism + Functional Programming = BSML n Bulk Synchronous Parallelism : n n Scalability Portability Simple cost model Functional Programming : n n n High level features (higher order functions, pattern matching, concrete types, etc. ) Programs proofs Safety of the environment F. Loulergue Pa. CT 2003 4

Bulk Synchronous Parallelism T(s) = (max 0 i<p wi) + h g + l F. Loulergue Pa. CT 2003 6

The BSMLlib Library n n n Bulk Synchronous Parallel ML library for the Objective Caml language operations on a parallel data structure Abtract type: par access to BSP parameters: bsp_p: unit int bsp_p() = number of processes F. Loulergue Pa. CT 2003 7

Creation of Parallel Vectors n mkpar: (int ) par (mkpar f ) (f 0) (f 1) … f (p-1) Cost: (max 0 i<p wi) F. Loulergue Pa. CT 2003 8

Pointwise Parallel Application n apply: ( ) par par apply f 0 f 1 … fp-1 v 0 v 1 … vp-1 f 0 v 0 f 1 v 1 F. Loulergue = … Pa. CT 2003 fp-1 vp-1 9

Communication Operation: Put type option = None | Some of put: (int option) par 0 1 2 None Some v 2 None put 3 None 0 None Some v 5 None Some v 3 None Some v 1 Some v 4 None v = Some None 1 None 2 3 None Some v 1 None Some v 3 Some v 4 Some v 5 None None Cost: (max 0 i<p wi) + h g + l F. Loulergue Pa. CT 2003 10

Global Conditional if vec at n then … else … if n b 0 b 1 … true … fp-1 at n then e 1 else e 2 = e 1 Cost: (p-1) g + l F. Loulergue Pa. CT 2003 11

Implementations of BSMLlib n BSMLlib v 0. 1 : n n O. Ballereau, G. Hains, F. Loulergue Ocaml + BSPlib End of 1999 BSMLlib v 0. 2 : n n n F. Loulergue Frédéric Gava, Xavier Leroy, Frédéric Loulergue Ocaml + MPI Available (http: //bsmllib. free. fr) Pa. CT 2003 12

Examples let replicate x = mkpar(fun pid->x) n (* bcast: int->’a par *) let bcast n vec = let tosend=mkpar(fun i v dst -> if i=n then Some v else None) in let recv=put(apply tosend vec) in apply (replicate no. Some) (apply recv (replicate n)) n F. Loulergue Pa. CT 2003 13

Problems with the nesting of parallel vectors F. Loulergue Pa. CT 2003

Example (1) let example 1 = mkpar(fun pid->bcast pid vec) (* example 1: t par *) F. Loulergue Pa. CT 2003 15

Example (2) let example 2 = mkpar(fun pid-> let this=mkpar(fun i->i) in pid) (* example 2: int par *) F. Loulergue Pa. CT 2003 16

Example (3) (* fst: ’a *’b ->’a *) let fst = fun (a, b) -> a n two usual values: fst(1, 2) n two parallel values: fst(mkpar(fun i->i), mkpar(fun i->i)) n parallel and usual: fst (mkpar(fun i->i), 1) n usual and parallel: fst (1, mkpar(fun i->i)) F. Loulergue Pa. CT 2003 17

Example (4) let v 1 x=mkpar(fun pid -> pid) and v 2 x=put(mkpar(fun i s-> 1+s))in let c 1 x = ((v 1 x), 1) and c 2 x = ((v 2 x), 2) in mkpar(fun pid -> if pid<(nproc/2) then snd (c 1()) else snd (c 2())) F. Loulergue Pa. CT 2003 18

Overview of the type system F. Loulergue Pa. CT 2003

Types F. Loulergue Pa. CT 2003 20

Constraints F. Loulergue Pa. CT 2003 21

Types of some predefined expressions F. Loulergue Pa. CT 2003 22

Typing judgments in the type environment E (which binds variables to types) the expression e has the type [t/C]. F. Loulergue Pa. CT 2003 23

Some Rules F. Loulergue Pa. CT 2003 24

Conclusion and Future Work n Conclusion n A Polymorphic Type to avoid the nesting of parallel vectors of BSML Implementation for a mini-BSML Future Work n n n F. Loulergue Sum types Imperative features Full implementation Pa. CT 2003 25