MPI Portable Parallel Programming for Scientific Computing William

MPI: Portable Parallel Programming for Scientific Computing William Gropp Rusty Lusk Debbie Swider Rajeev Thakur

Portable Parallel Programming • Why Parallel? For performance: – Realtime needs (weather prediction) – Large memory (throughput) • Why Portable? – Lifetime of code >> machine – Performance through algorithms

The Message Passing Model • Parallel programs consist of cooperating processes, each with its own memory • Processes send data to one another as messages • Messages may have tags that may be used to sort messages • Messages may be received in any order

What is MPI? • Message-passing model • Standard (specification) – Many implementations – MPICH most widely used • Two phases: – MPI 1: Traditional message-passing – MPI 2: Remote memory, parallel I/O, and dynamic processes

Why was MPI needed? • Software crisis in parallel computing – Each vendor provided their own, different interface – No “critical mass” of users • Enabling libraries (code sharing) • Turnkey parallel applications – CFD, pharmaceutical design, etc.

Quick Tour of MPI • • • Point-to-point Collective Process groups and topology Profiling Other

Point-to-point • Send/Receive A(10) MPI_Send( A, 10, MPI_DOUBLE, 1, …) • Datatype – Basic for heterogeneity – Derived for non-contiguous • Contexts – Message safety for libraries • Buffering – Robustness and correctness B(20) MPI_Recv( B, 20, MPI_DOUBLE, 0, … )

Collective • Process groups – Collections of cooperating processes – Hierarchical algorithms need nested collections • Categories – Communication: Broadcast data – Computation: Global sum – Synchronization: Barrier

New MPI-2 Features • • Remote Memory Parallel I/O Dynamic Process Threads

MPI in the World • Libraries – At ANL: PETSc, SUMMA 3 d, etc. – Others: Aztec, PESSL, Sca. LAPACK, FFTW, etc. • Applications – – – – Electromagnetic Scattering Groundwater pollution Stellar atmospheres Reactive flows QCD Navier-Stokes Convective flows Multi-physics in metal forming • Papers

MPI Research at MCS • • Portable performance I/O Programming Environment New hardware – Network/Memory interconnect (VIA) • New Programming Models – Threads – Remote memory

Learning More • MPI Books – Using MPI – MPI: The Complete Reference – Parallel Programming with MPI • Online Resources – http: //www. mcs. anl. gov/mpi – MPI Forum: http: //www. mpi-forum. org – Tutorials: http: //www. mcs. anl. gov/mpi/learning. html