Seeds Framework B WilkinsonClayton Ferner Seeds Framework ppt

Seeds Framework B. Wilkinson/Clayton Ferner Seeds. Framework. ppt Modification date August 15 2014 1

“Seeds” Parallel Grid Application Framework Some Key Features • Pattern-programming Java user interface (C++ version developed) • Self-deploys on computers, clusters, and geographically distributed computers. • Three development layers, basic, advanced and expert, exposing increasing detail. • We will use the basic level. http: //coit-grid 01. uncc. edu/seeds/ 2

Seeds programming Several standard patterns implemented including Workpool, Pipeline, All-to-all, Stencil, etc. Slaves Workpool Three phases: 1. Master diffuses data to slaves 2. Slaves performs computations 3. Master gathers results for slaves Programmer specifies what master and slave do, and what is transferred between them, without implementing low level message passing routines. Master Slaves Compute Diffuse Gather Master Message passing done by Seeds 3

User Program “Module” class Two classes: 1. “Module” class – diffuse, compute and gather methods and any other methods associated with application Diffuse Compute Gather 2. Run module “Bootstrap” class creates an instance of the module class and starts the framework and executes module pattern. Run module Bootstrap class 4

Seeds Workpool Diffuse. Data, Compute, and Gather. Data Methods Master Diffuse. Data Gather. Data. Map d Private variable total (answer) Returns d to each slave Data argument data Compute Slaves Data argument data Data. Map input Data. Map output d created in Diffuse. Data. output created in Compute Diffuse. Data, Compute and Gather. Data methods start with a capital letter although method names should not! 5

Data and Data. Map classes For implementation convenience two classes: • Data class used to pass data between master and slaves (Uses a “segment” number to keep track of packets, see later). • Data. Map class inside compute method Data. Map is a subclass of Data and so allows casting. Data. Map methods • put (String, data) – puts data into Data. Map identified by string • get (String) – gets stored data identified by string Data. Map extends Java Hash. Map which implement a Map, see http: //doc. java. sun. com/Doc. Web/api/java. util. Hash. Map 6

Module class public Data Diffuse. Data (int segment) { Data. Map<String, Object> d =new Data. Map<String, Object>(); input Data = …. d. put(“name_of_inputdata", input. Data); return d; } Data By framework segment used by Seeds to keep track of where to put results cast into a Data. Map public Data Compute (Data data) { Data. Map<String, Object> input = (Data. Map<String, Object>)data; //data produced by Diffuse. Data() Data. Map<String, Object> output = new Data. Map<String, Object>(); //output returned to gatherdata input. Data = input. get(“name_of_inputdata”); … // computation output. put("name_of _results", results); return output; By framework } // to return to Gather. Data() Gather. Data gives back Data object with a segment number public void Gather. Data (int segment, Data dat) { Data. Map<String, Object> out = (Data. Map<String, Object>) dat; outdata = out. get (“name_of_results”); result … // aggregate outdata from all the worker nodes. result a private variable } 7

Question Will a class field modified in the Diffuse. Data or Gather. Data methods be updated with the same values as in the Compute method? Answer NO. The two methods are running on different JVMs (and different nodes) PP-2. 8

Seeds Implementations Three Java versions developed: • Full JXTA P 2 P version intended for a cluster and a fully distributed system (grid system). Requires an Internet connection. • JXTA P 2 P version not needing an external network but otherwise identical, suitable for testing on a single computer. • Multicore (thread-based) version specifically a single multicore computer. Multicore version much faster execution on a single computer. Only difference is minor coding changes in bootstrap class. 9

Bootstrap class JXTA P 2 P version This code deploys framework and starts execution of pattern Different patterns have similar code package edu. uncc. grid. example. workpool; import java. io. IOException; import net. jxta. pipe. Pipe. ID; import edu. uncc. grid. pgaf. Anchor; import edu. uncc. grid. pgaf. Operand; import edu. uncc. grid. pgaf. Seeds; import edu. uncc. grid. pgaf. p 2 p. Types; public class Run. Monte. Carlo. Pi. Module { public static void main(String[] args) { try { My. Module pi = new My. Module(); Seeds. start( "/path/to/seeds/seed/folder" , false); Pipe. ID id = Seeds. start. Pattern(new Operand( (String[])null, new Anchor("hostname", Types. Data. Flow. Roll. SINK_SOURCE), pi )); System. out. println(id. to. String() ); Seeds. wait. On. Pattern(id); Seeds. stop(); System. out. println( "The result is: " + pi. get. Pi() ) ; } catch (Security. Exception e) { e. print. Stack. Trace(); } catch (IOException e) { e. print. Stack. Trace(); } catch (Exception e) { e. print. Stack. Trace(); } } 10 }

Bootstrap class Multicore version • Multicore version • Much faster on a multicore platform • Thread based • Bootstrap class does not need to start and stop JXTA P 2 P. Seeds. start() and Seeds. stop() not needed. Otherwise user code similar. public class Run. Monte. Carlo. Pi. Module { public static void main(String[] args) { try { My. Module pi=new My. Module(); Thread id = Seeds. start. Pattern. Multicore( new Operand( (String[])null, new Anchor( args[0], Types. Data. Flow. Role. SINK_SOURCE), pi ), 4); id. join(); System. out. println( "The result is: " + pi. get. Pi() ) ; } catch (Security. Exception e) { e. print. Stack. Trace(); } catch (IOException e) { e. print. Stack. Trace(); } catch (Exception e) { e. print. Stack. Trace(); } } }

Measuring Time Can instrument code in the bootstrap class: public class Run. My. Module { public static void main (String [] args ) { try{ long start = System. current. Time. Millis(); My. Module m = new My. Module(); Seeds. start(. ); Pipe. ID id = ( … ); Seeds. wait. On. Pattern(id); Seeds. stop(); long stop = System. current. Time. Millis(); double time = (double) (stop - start) / 1000. 0; System. out. println(“Execution time = " + time); } catch (Security. Exception e) { … …

Compiling/executing • Can be done on the command line (ant script provided) or through an IDE (Eclipse) 13

Examples of applications using Workpool Pattern 1. Computing p by the Monte Carlo method 14

Monte Carlo Methods A so-called “embarrassingly parallel” computation as it decomposes into obviously independent tasks that can be done in parallel without any task communications during the computation. Monte Carlo methods use random selections. For parallelizing Monte Carlo code, must address best way to generate random numbers in parallel. 15

Calculate p using the Monte Carlo method Circle formed within a 2 x 2 square. Ratio of area of circle to square given by: Points within square chosen randomly. Score kept of how many points happen to lie within circle. Fraction of points within circle will be , given sufficient number of randomly selected samples. 16

Typically one quadrant used. One quadrant can be described by integral: Random pairs of numbers, (xr, yr) generated, each between 0 and 1. Counted as in circle if 17

Alternative (better) Monte Carlo Method (Not used here) Generate random values of x to compute f(x) Sum values of f(x): where xr are randomly generated values of x between x 1 and x 2. Monte Carlo method very useful if the function cannot be integrated numerically (maybe having a large number of variables) 3. 19

Workpool implementation Slaves Compute Return number of 1000 random points inside arc of circle inside Send by starting seed for random sequence to each slave seed Aggregate answers Gather. Data Diffuse. Data Master Compute node Source/sink 19

Seeds Monte Carlo code Monte. Carlo. Pi. Module. java Diffuse. Data Method (Required to be implemented) public Data Diffuse. Data (int segment) { Data. Map<String, Object> d =new Data. Map<String, Object>(); d. put("seed", R. next. Long()); return d; // returns a random seed for each job unit } 20

Compute Method (Required to be implemented) public Data Compute (Data data) { Data. Map<String, Object> input = (Data. Map<String, Object>)data; Data. Map<String, Object> output = new Data. Map<String, Object>(); Long seed = (Long) input. get("seed"); // get random seed Random r = new Random(); r. set. Seed(seed); Long inside = 0 L; for (int i = 0; i < Double. Data. Size ; i++) { double x = r. next. Double(); double y = r. next. Double(); double dist = x * x + y * y; if (dist <= 1. 0) { ++inside; } } output. put("inside", inside); // to return to Gather. Data() return output; } 21

Gather. Data Method (Required to be implemented) public void Gather. Data (int segment, Data dat) { Data. Map<String, Object> out = (Data. Map<String, Object>) dat; Long inside = (Long) out. get("inside"); total += inside; // aggregate answer from all the worker nodes. } 22

get. Data. Count Method (Required to be implemented) public int get. Data. Count() { return random_samples; } Set number of data “envelopes” sent from master by Diffuse. Data to slaves, in this case number of “seeds”. (Number of physical slaves processors might be different. ) Initialized in: initialize. Module(String[ ] args) { random_samples = 3000; } 23

Method to compute p result (used in bootstrap module) public double get. Pi() { // returns value of pi based on all workers double pi = (total / (random_samples * Double. Data. Size)) * 4; return pi; } 24

Complete module class Monte. Carlo. Pi. Module package edu. uncc. grid. example. workpool; import java. util. Random; import java. util. logging. Level; import edu. uncc. grid. pgaf. datamodules. Data. Map; import edu. uncc. grid. pgaf. interfaces. basic. Workpool; import edu. uncc. grid. pgaf. p 2 p. Node; public class Monte. Carlo. Pi. Module extends Workpool { private static final long serial. Version. UID = 1 L; private static final int Double. Data. Size = 1000; double total; int random_samples; Random R; public Monte. Carlo. Pi. Module() { R = new Random(); } public void initialize. Module(String[] args) { total = 0; Node. get. Log(). set. Level(Level. WARNING); // reduce verbosity for logging random_samples = 3000; // set # of random samples } } public Data Compute (Data data) { // input gets the data produced by Diffuse. Data() Data. Map<String, Object> input = (Data. Map<String, Object>)data; Data. Map<String, Object> output = new Data. Map<String, Object>(); Long seed = (Long) input. get("seed"); // get random seed Random r = new Random(); r. set. Seed(seed); Long inside = 0 L; for (int i = 0; i < Double. Data. Size ; i++) { double x = r. next. Double(); double y = r. next. Double(); double dist = x * x + y * y; if (dist <= 1. 0) { ++inside; } } output. put("inside", inside); // store partial answer to return to Gather. Data() return output; // output will emit partial answers done by this method } public Data Diffuse. Data (int segment) { Data. Map<String, Object> d =new Data. Map<String, Object>(); d. put("seed", R. next. Long()); return d; // returns a random seed for each job unit } public void Gather. Data (int segment, Data dat) { Data. Map<String, Object> out = (Data. Map<String, Object>) dat; Long inside = (Long) out. get("inside"); total += inside; // aggregate answer from all the worker nodes. } public double get. Pi() { // returns value of pi based on the job done by all the workers double pi = (total / (random_samples * Double. Data. Size)) * 4; return pi; } public int get. Data. Count() { return random_samples; } 25

Bootstrap class (Multicore version). . . public class Run. Monte. Carlo. Pi. Module { public static void main(String[] args) { try { Monte. Carlo. Pi. Module pi = new Monte. Carlo. Pi. Module(); Seeds. start( "/path/to/seeds/seed/folder" , false); Pipe. ID id = Seeds. start. Pattern(new Operand( (String[])null, new Anchor("hostname", Types. Data. Flow. Roll. SINK_SOURCE), pi)); System. out. println(id. to. String() ); Seeds. wait. On. Pattern(id); Seeds. stop(); System. out. println( "The result is: " + pi. get. Pi() ) ; } catch (Security. Exception e) {. . . 26

Discussion • Does anyone see a potential flaw in the code (clue: random number generation) 27

Workpool pattern 2. Matrix addition and multiplication very easy to parallelize as each result value independent of other result values. 28

Matrix Addition, C = A + B Add corresponding elements of each matrix to form elements of result matrix. Given elements of A as ai, j and elements of B as bi, j, each element of C computed as: Add A B C Easy to parallelize – each processor computes one C element or group of C elements 29

Workpool Implementation Slave computation Adds one row of A with one row of B to create one row of C (rather than each slave adding single elements) Add A B C Note generally we want the Computation/Communication ratio as large as possible. Here it is O(1)! 30

Workpool implementation Slaves (one for each row) Return one row of C C A B Send one row of A and B to slave Master Following example 3 x 3 arrays and 3 slaves Compute node Source/sink 31

package edu. uncc. grid. example. workpool; Matrix. Add. Module. java import … Continues on several sides public class Matrix. Add. Module extends Workpool { private static final long serial. Version. UID = 1 L; int[][] matrix. A; int[][] matrix. B; int[][] matrix. C; public Matrix. Add. Module() { In this example matrix. C = new int[3][3]; matrices are 3 x 3 } public void init. Matrices(){ matrix. A = new int[][]{{2, 5, 8}, {3, 4, 9}, {1, 5, 2}}; Some initial values matrix. B = new int[][]{{2, 5, 8}, {3, 4, 9}, {1, 5, 2}}; } public int get. Data. Count() { Required method. Number of return 3; data objects (Slaves) } public void initialize. Module(String[] args) { Node. get. Log(). set. Level(Level. WARNING); } 32

Diffuse. Data method public Data Diffuse. Data(int segment) { int[] row. A = new int[3]; int[] row. B = new int[3]; Data. Map d returned are pairs of string key and associated array Data. Map<String, int[]> d =new Data. Map<String, int[]>(); int k = segment; for (int i=0; i<3; i++) { row. A[i] = matrix. A[k][i]; row. B[i] = matrix. B[k][i]; } d. put("row. A", row. A); d. put("row. B", row. B); return d; } segment variable used to select rows Copy one row of A and one row of B into row. A, row. B to be sent to slaves row. A and row. B put in d Data. Map to send to slaves 33

Compute method public Data Compute(Data data) { int[] row. C = new int[3]; Data. Map<String, int[]> input = (Data. Map<String, int[]>)data; Data. Map<String, int[]> output = new Data. Map<String, int[]>(); int[] row. A = (int[]) input. get("row. A"); int[] row. B = (int[]) input. get("row. B"); for (int i=0; i<3; i++) { row. C[i] = row. A[i] + row. B[i]; } output. put("row. C", row. C); return output; } Get two rows from data received Add rows Put result row into output with key to be sent back to master 34

Gather. Data method Note segment variable and Data from slave public void Gather. Data(int segment, Data dat) { Data. Map<String, int[]> out = (Data. Map<String, int[]>) dat; int[] row. C = (int[]) out. get("row. C"); for (int i=0; i<3; i++) { matrix. C[segment][i]= row. C[i]; } Get C row sent from slave Place row into result matrix Segment variable associated with Data used to choose correct row } 35

Bootstrap class Multicore version public class Run. Monte. Carlo. Pi. Module { public static void main(String[] args) { try { long start = System. current. Time. Millis(); Matrix. Add. Module m = new Matrix. Add. Module(); m. init. Matrices(); Thread id = Seeds. start. Pattern. Multicore( new Operand( (String[])null, new Anchor( args[0], Types. Data. Flow. Role. SINK_SOURCE), pi ), 4); id. join(); long stop = System. current. Time. Millis(); double time = (double) (stop - start) / 1000. 0; System. out. println("Execution time = " + time); m. print. Result(); } catch … 36

Matrix Multiplication Sequential code to compute A x B square (n x n matrices) for (i = 0; i < n; i++) // for each row of A for (j = 0; j < n; j++) { // for each column of B c[i][j] = 0; for (k = 0; k < n; k++) c[i][j] = c[i][j] + a[i][k] * b[k][j]; } Requires n 3 multiplications and n 3 additions. Sequential time complexity of O(n 3). Very easy to parallelize as each result independent 37

Workpool implementation With one slave computing one element of result: Slaves (one for each element of result) Return one element of C C A Send one row of A and one column of B to slave B Compute node Master Following example 3 x 3 arrays and 9 slaves Source/sink 38

package edu. uncc. grid. example. workpool; Matrix. Add. Module. java import … Continues on several sides public class Matrix. Add. Module extends Workpool { private static final long serial. Version. UID = 1 L; int[][] matrix. A; int[][] matrix. B; int[][] matrix. C; public Matrix. Add. Module() { In this example matrix. C = new int[3][3]; matrices are 3 x 3 } public void init. Matrices(){ matrix. A = new int[][]{{2, 5, 8}, {3, 4, 9}, {1, 5, 2}}; Some initial values matrix. B = new int[][]{{2, 5, 8}, {3, 4, 9}, {1, 5, 2}}; } public int get. Data. Count() { Required method. Number of return 9; data objects (Slaves) } public void initialize. Module(String[] args) { Node. get. Log(). set. Level(Level. WARNING); } 39

Diffuse. Data method public Data Diffuse. Data(int segment) { int[] row. A = new int[3]; int[] col. B = new int[3]; Data. Map d returned are pairs of string key and associated array Data. Map<String, int[]> d =new Data. Map<String, int[]>(); segment variable used to select element in A and B int a=segment/3, b = segment%3 ; for (int i=0; i<3; i++) { row. A[i] = matrix. A[a][i]; Copy one row of A and one column of col. B[i] = matrix. B[i][b]; B into row. A, col. B to be sent to slaves } d. put("row. A", row. A); row. A and col. B put in d Data. Map to d. put(“col. B", col. B); send to slaves return d; } 40

Note on mapping rows and columns to segments segment 0 segment 1 segment 2 segment 3 segment 4 segment 5 segment 6 segment 7 segment 8 Arow Bcol 0 0 0 1 0 2 1 0 1 1 1 2 2 0 2 1 2 2 int Arow =segment/3; Int Bcol = segment%3; 41

Compute method public Data Compute(Data data) { int[] row. C = new int[3]; Data. Map<String, int[]> input = (Data. Map<String, int[]>)data; Data. Map<String, Integer> output = new Data. Map<String, Integer>(); int[] row. A = (int[]) input. get("row. A"); int[] col. B = (int[]) input. get(“col. B"); int out = 0; for (int i=0; i<3; i++) { out += row. A[i]*col. B[i]; } output. put(“out", out); return output; } Get two rows from data received Matrix multiplication, one result Put result into output with key to be sent back to master 42

Gather. Data method Note segment variable and Data from slave public void Gather. Data(int segment, Data dat) { Data. Map<String, Integer> out = (Data. Map<String, Integer>) dat; int answer = out. get("out"); Get result sent from slave* int a=segment/3, b=segment%3; Place element into result matrix Segment variable associated with matrix. C[a][b]= answer; Data used to choose correct row } * Cast from Integer to int not necessary 43

Workpool F(x) Numerical integration Start End x Slaves (one for each partition) Area Send start and end for partition to slave Return computed area under curve Start End Compute node Master Source/sink

Questions 45