Message Passing Computing Lecture 2 Message Passing Helvi

Message Passing Computing Lecture 2 Message Passing Helvi Hartmann FIAS 1 Inverted CERN School of Computing, 23 -24 February 2015 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Cluster Computing parallel computing on systems with distributed memory 2 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing ZMQ, PVM, MPI, HPX, Open. MP Any questions? 3 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing What Methods of Message Passing do you know? 4 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Inter Process Communication File 5 From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Inter Process Communication File Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory 6 From every process accessible information Processes access same data in memory i. CSC 2015, Helvi Hartmann, FIAS Open. MP

Message Passing Computing Inter Process Communication File Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket 7 From every process accessible information Processes access same data in memory Data is send to specific sockets i. CSC 2015, Helvi Hartmann, FIAS Open. MP ZMQ

Message Passing Computing Inter Process Communication File From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket Processes access same data in memory Data is send to specific sockets Open. MP ZMQ PVM Message Passing 8 Data is send to specific sockets i. CSC 2015, Helvi Hartmann, FIAS MPI

Message Passing Computing Inter Process Communication File From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket Processes access same data in memory Data is send to specific sockets Open. MP ZMQ PVM Message Passing Data is send to specific sockets Remote Procedure Call 9 Processes outsource work to subroutines i. CSC 2015, Helvi Hartmann, FIAS MPI ZMQ

Message Passing Computing Inter Process Communication File From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket Processes access same data in memory Data is send to specific sockets Open. MP ZMQ PVM Message Passing Data is send to specific sockets Remote Procedure Call 10 Processes outsource work to subroutines i. CSC 2015, Helvi Hartmann, FIAS MPI ZMQ

Message Passing Computing Open. MP 11 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Inter Process Communication File From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket Processes access same data in memory Data is send to specific sockets Open. MP ZMQ PVM Message Passing Data is send to specific sockets Remote Procedure Call 12 Processes outsource work to subroutines i. CSC 2015, Helvi Hartmann, FIAS MPI ZMQ

Message Passing Computing Socket Programming 13 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing ZMQ • automatically reconnects to peers • queues messages at sender and receiver • routes messages • handles sockets errors • does all I/O in background threads • lock free techniques for communication 14 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing ZMQ – Socket Types Request/Reply blocking 15 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing ZMQ – Socket Types Push/Pull nonblocking 16 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing ZMQ – Socket Types Publish/Subscribe 17 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Inter Process Communication File From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket Processes access same data in memory Data is send to specific sockets Open. MP ZMQ PVM Message Passing Data is send to specific sockets Remote Procedure Call 18 Processes outsource work to subroutines i. CSC 2015, Helvi Hartmann, FIAS MPI ZMQ

Message Passing Computing Message Passing 19 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing PVM Process 0 Master Process 0 Process 1 Process 2 20 Slave Process 1 Process 2 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI node 0 Processes ressource manager node 1 Process 1 21 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI - Blocking Processes 22 Processes send receive i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI - Blocking Synchronous Send (MPI_Ssend) 23 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI - Blocking Buffered Send (MPI_Bsend) 24 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI - Blocking 25 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI - Blocking Synchronous Send (MPI_Ssend) 26 i. CSC 2015, Helvi Hartmann, FIAS Buffered Send (MPI_Bsend)

Message Passing Computing MPI - Blocking Processes send receive function returns immediately 27 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI - Nonblocking 28 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI - Nonblocking 29 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI – Collectives Broadcast 30 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing MPI – Collectives Scatter 31 i. CSC 2015, Helvi Hartmann, FIAS Gather

Message Passing Computing Inter Process Communication File From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket Processes access same data in memory Data is send to specific sockets Open. MP ZMQ PVM Message Passing Data is send to specific sockets Remote Procedure Call 32 Processes outsource work to subroutines i. CSC 2015, Helvi Hartmann, FIAS MPI ZMQ

Message Passing Computing Futures locality 1 ? locality 1 locality 2 ? future 33 Time i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Futures locality 1 ? locality 1 2 34 3 locality 2 locality 3 locality 4 ? ? ? 4 Time i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Inter Process Communication File From every process accessible information Signal Asynchrounous notification of an event Pipe output of each process streams directly as input for another process Shared Memory Socket Processes access same data in memory Data is send to specific sockets Open. MP ZMQ PVM Message Passing Data is send to specific sockets Remote Procedure Call 35 Processes outsource work to subroutines i. CSC 2015, Helvi Hartmann, FIAS MPI ZMQ

Message Passing Computing 36 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing 05. 11. 2014: http: //creatia 2013. wordpress. com/2013/03/12/dna-is-like-a-computer-program-butfar-more-advanced-than-any-software-weve-ever-created-bill-gates/ http: //www. lhc-facts. ch/index. php? page=datenverarbeitung http: //www. computerhistory. org/fellowawards/hall/bios/Gene, Amdahl/ http: //www. heise. de/newsticker/meldung/AMD-bringt-neuen-Notebook-Prozessor. Llano-1259681. html http: //en. wikipedia. org/wiki/TBR 1 http: //www. adpic. de/lizenzfreie_bilder/Geld%20_und_%20 Boerse/Euro/goldener_g itter_globus_253863. html 37 i. CSC 2015, Helvi Hartmann, FIAS