Message Passing Computing Lecture 1 High Performance Computing

Message Passing Computing Lecture 1 High Performance Computing Helvi Hartmann FIAS 1 Inverted CERN School of Computing, 23 -24 February 2015 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing What is High Performance Computing? 2 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Cluster Computing parallel computing on systems with distributed memory 3 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Why do we need it? 4 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Example: Weather Forecast 5 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Grand Challenges 6 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Grand Challenges 7 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Big Data 8 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Why not make a single fast computer? processing 1016 operation/s instead of 109 operation/s 9 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Memory Wall 1 instruction in 4 ns memory 27 cycles + 40 ns = 148 ns 10 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Memory Wall 1 instruction in 4 ns L 1 Cache 4 cycles = 16 ns L 2 Cache 12 cycles = 48 ns L 3 Cache 27 cycles = 108 ns memory = 148 ns 11 Caches —> located directly next to the CPU to reduce access time - small space available - expensive in power usage i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Moore‘s Law Amount of transistor on a chip will double every second year —> „computerspeed doubles every second year“ Use of multiprocessors 12 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Microarchitecture Pipelining Multithreading Caching Prefetching Out-of-order execution Vectorization Only small gains as compared to if Moore‘s law remained valid 13 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Shared Memory 14 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Shared Memory • Parallel Processing • Memory Wall 15 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Shared Memory • Parallel Processing • Memory Wall • Cache Coherency 16 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing 17 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Shared Memory • Parallel Processing • Memory Wall • Cache Coherency • Expensive!!! 18 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing What is possible except improving the Microarchitecture? 19 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Cluster Computing 20 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Distributed Memory 21 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Shared Memory • Cheap • Standard processor very powerful • Scalable • High Latencies 22 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing High Latencies? Detour: Network Topologies 23 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing 24 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Does it work any easier? 25 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing 3 D 26 i. CSC 2015, Helvi Hartmann, FIAS switches

Message Passing Computing Parallelization 27 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Amdahl‘s Law sequential part limits speedup significantly 28 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Parallelization While data points increase by a factor 1000 communication grows by 10 29 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Parallelization 30 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing History of parallel computing 31 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing History of Parallel Computing Beowulf Cluster 32 1994 NASA: Goddard Space Flight Center calculation of 2 D streaming properties 16 Linux PC’s i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing 33 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing History of Parallel Computing Linux free and open software stability and portability PVM free and open standard 34 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing History of Parallel Computing SETI@home search for extraterrestrial intelligence analyse data of radioteleskopes for extraterrestrial signals 1999 5 million people shared computing time April 2005 60 TFlops, fastest computer 70, 7 TFlops, second fastest 51, 8 TFlops 35 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing CBM - FLES CBM untriggered Experiment Datarate of 1 TB/s 1000 input nodes 36 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing Conclusion 37 i. CSC 2015, Helvi Hartmann, FIAS

Message Passing Computing 05. 11. 2014: http: //creatia 2013. wordpress. com/2013/03/12/dna-is-like-a-computer-program-butfar-more-advanced-than-any-software-weve-ever-created-bill-gates/ http: //www. lhc-facts. ch/index. php? page=datenverarbeitung http: //www. computerhistory. org/fellowawards/hall/bios/Gene, Amdahl/ http: //www. heise. de/newsticker/meldung/AMD-bringt-neuen-Notebook-Prozessor. Llano-1259681. html http: //en. wikipedia. org/wiki/TBR 1 http: //www. adpic. de/lizenzfreie_bilder/Geld%20_und_%20 Boerse/Euro/goldener_g itter_globus_253863. html 38 i. CSC 2015, Helvi Hartmann, FIAS