Bulk Synchronous Parallel Computing Trevor Schaub Jim Sellers

Bulk Synchronous Parallel Computing Trevor Schaub Jim Sellers This presentation was prepared for Professor Stefan Dobrev in partial fulfillment of the 1 requirements for the course CSI 4140

Outline n n Model description Why it is interesting/important Relationship to real life Some basic techniques, simple sample program to demonstrate how the approach to algorithm design differs from PRAM and MPP 2

Why do we need change? n n Diverse parallel computers and models Shared mem. vs. message passing, no clear winner Use network locality vs. don’t Conclusion: parallel code is often computer specific. Need unifying model 3

Why BSP? n Provides consistent, general framework to develop scaleable software n n n Similar to OO – a way of thinking and writing programs (paradigm) Not system specific Proceed in “supersteps” n n Decouples communication and synchronization The deadlock problem is minimized 4

What is BSP? n n n BSP is a process of dividing a task into steps called supersteps Each process performs the task by doing the supersteps in order The fundamental concept is the superstep itself, and it’s structure 5

What’s a superstep? 1. 2. 3. Processes perform as much as possible using local data Communication and synchronization Move on to next superstep … 6

Dissection of a superstep Threads/Processes T I M Local computation E Global Communication Synchronization 7

Model description n What are the benefits of a superstep? n Easier debugging n n Removes problem of deadlock n n Can look at supersteps in isolation Separate computation from communication Allows reasoning of the correctness of the code nearly as easy as sequential code. n Easier shift for programmers used to sequential programming 8

Relationship to real life n Is BSP in use today? n n Yes, sometimes without even realizing it. When should you use BSP? n n Becomes more useful with more communication Benefits vary with the problem 9

Basic techniques n n Identify the sections that can be that need communication or synchronization Modify program so that sync. happens after communication n Separate ones are grouped where possible Before each communication group there is a global barrier The code between the end of each communication group becomes a superstep 10

Example: n See code example 11

References n n n n n http: //approximity. com/papers/ptfopt/node 31. html http: //www. cs. ucf. edu/csdept/faculty/goudreau/cop 5937_fall 96/ COP 5937. html http: //web. comlab. ox. ac. uk/oucl/research/highlights/bsp_compu ting. html http: //www. math. uu. nl/people/bisselin/pas 1_prev. html http: //www. math. ruu. nl/people/bisselin/software. html http: //web. comlab. ox. ac. uk/oucl/work/bill. mccoll/oparl. html http: //www. cs. hmc. edu/courses/2001/spring/cs 156/html 12/slide s 12. pdf http: //www. cag. lcs. mit. edu/bayanihan/papers/javapdc 99/html/n ode 2. html http: //www. byte. com/art/9611/sec 5/art 5. htm 12

13