http comp is uec ac jpyoshinagalabyoshinagadp 2 html

内容 • 分散・並列処理計算機における相互結合ネット ワークとその上でのメッセージ・ルーティング技 法などについて学ぶ • 資料　 http: //comp. is. uec. ac. jp/yoshinagalab/yoshinaga/dp 2. html • http: //www. gap. upv. es/slides/appendix. E. html • http: //booksite. mkp. com/9780123838728/references. php appendix_f. pdf (P. 118, 2 MB) • TA: 溝田 敦也　amizota@comp. is. uec. ac. jp 　　　　 ＮＣ論２ 2

References • T. M. Pinkston and J. Duato: Interconnection Networks, Appendix E in Computer Architecture: A Quantitative Approach, 4 th Edition, Morgan Kaufmann publishers (2006). • 5 th Edition, Morgan Kaufmann publishers (2011). • J. Duato, S. Yalamanchili, L. Ni: Interconnection Networks - an Engineering Approach-, 第 2版, Morgan Kaufmann publishers (2003) • 富田眞治：　並列コンピュータ、昭晃堂（1996） • W. D. Dally, B. Towles: Principles and Practices of Interconnection Networks, Morgan Kaufmann publishers (2003) 3

What is an interconnection Network? • It is a programmable system that transports data between terminals, such as processors and memory. • It is programmable in the sense that it makes different connections at different points. • It is a system because it is composed of many components: buffers, channels, switches, and controls that works together to deliver data. ＮＣ論２ 4

Interconnection Network (1/2) Interconnection Network P P P M Multicomputer ＮＣ論２ 5

Interconnection Network (2/2) P P P Interconnection Network M M M UMA type shared memory multiprocessor It is also called dance-hall architecture. ＮＣ論２ 6

Trend • Its performance is increasing with processor performance at a rate of 50% per year. • Communication is a limiting factor in the performance of many modern systems. • Buses have been unable to keep up with the bandwidth demand, and point-to-point interconnection networks are rapidly taking over. ＮＣ論２ 7

Computer Classifications (%) 2016/06 2015/06 2014/06 MPP 13. 8 13. 2 14. 6 Cluster 86. 2 86. 8 85. 4 Others 0. 0 http: //www. top 500. org/ share of the TOP 500 June, 2016 – June, 2014 ＮＣ論２ 8

Examples of clusters Processors Accelerator Interconnect Tianhe-2 (天河 2号） China 2013 Intel Xeon E 5 -2692 12 C 2. 2 GHz× 2 × 16 K Xeon Phi 31 S 1 P (57 cores)× 3 × 16 K TH Express-2 (proprietary) Fat tree Tsubame 2. 5 Tokyo Tech. 2013 Xeon X 5670 2. 93 GHz× 2 × 1, 408 NVIDIA Kepler K 20 x × 3× 1, 048 Infiniband QDR (40 Gbps) × 2 Fat tree ＮＣ論２ 9

Examples of MPPs Node Topology #core Rmax K computer @RIKEN Fujitsu 2011 SPARC 64 VIIIfx 2 GHz (16 GFlops× 8 -core) 6 D mesh/ 3 D torus Tofu interconnect 80 K-node x 8 -core = 640 K-core 10. 51 PFlops 7, 890 KW Sunway Taihu. Light 2016 SW 26010 1. 45 GHz (11. 6 GFlops× 260 -core) 5 -level hierarchical Sunway　 interconnect 40, 960 nodes 125. 4 PFlops 15, 371 KW ＮＣ論２ 10

Other Networks of Supercomputers • Titan/Cray XK 7 (2012): 3 D torus, proprietary Gemini interconnect • Sequoia/BG Q (2011): 5 D torus, proprietary IBM Sea. Star • Pleiades / NASA (2011): partial 11 D hypercube topology with IB QDR/DDR • Red Sky/ Sandia National Lab. (2010): 3 D torus (12 bristled node) with IB QDR switches • IBM Roadrunner (2009): fat-tree with IB DDR • Earth Simulator 2 / NEC SX-9 E (2009): Fat-Tree (64 GB/s/cpu, 8 -CPU/node, 160 nodes) ＮＣ論２ 11

Architecture vs. software memory programming UMA (SMP) shared Open. MP NUMA (MPP) distributed (not shared) MPI (Message Passing Interface) ＮＣ論２ 12

Network Design (1/3) • Performance: latency and throughput (bandwidth) • Scalability: #processors vs. network, memory, I/O bandwidth • Incremental expandability: small to maximum size • Partitionability: netwrok may be partitioned for several users ＮＣ論２ 13

Network Design (2/3) • Simplicity: simple design, higher clock frequency, easy to use • Distance span: smaller system is preferred for noise and cable delay, etc. • Physical constraints: packaging (pin count), wiring(wire length), and maintenance (power consumption) should meet physical limitation. ＮＣ論２ 14

Network Design (3/3) • Reliability: fault tolerant, reliable communication, hot swap • Expected workload: robust performance over a wade range of traffic conditions. • Cost: trade-offs between cost and performance. ＮＣ論２ 15

Classifiction of Interconnection Networks • Shared-Medium Networks – Local area networks (ethernet, token ring) – Backplane bus (e. g. SUN Gigaplane) • Direct Networks (router-based) – mesh, torus, hypercube, tree, … etc. • Indirect Networks (switch-based) • Hybrid Networks ＮＣ論２ 16

Shared-Medium Networks (LAN) • Arbitration that determines the mastership of the shared-medium network to resolve network access is needed. • The most well-known protocol is carrier-sense multiple access with collision detection (CSMA/CD). • Token bus and token ring pass a token from the owner which has the right to access the bus/ring and resolve nondeterministic waiting time. ＮＣ論２ 17

Shared-Medium Networks (Backplane bus) • It is commonly used to interconnect processor(s) and memory modules to provide SMP (Symmetrical Memory Processor) architecture. • It is realized by printed lines on a circuit board by discrete wiring. • Gigaplane in SUN Enterprise x 000 server(1996): 2. 6 GB/s, 256 bits data, 42 bits address, 83. 8 MHz clock. ＮＣ論２ 18

Direct (static) Networks • Consists of a set of nodes. • Each node is directly connected to a subset of other nodes in the network. • Examples: – 2 D mesh (intel Paragon), 3 D mesh (MIT J-Mahine) – 2 D torus (Fujitsu AP 3000), 3 D torus (Cray T 3 D, T 3 E) – Hypercube (CM 1, CM 2, n. CUBE) ＮＣ論２ 19

Mesh topology node 2 D 3 D ＮＣ論２ 20

Torus topology 2 D 3 D (4 -ary 2 -cube) (3 -ary 3 -cube) ＮＣ論２ 21

Hypercube (binary n-cube) 4 D (2 -ary 4 -cube) ＮＣ論２ 22

tree Binary tree fat tree ＮＣ論２ x tree 23

Hierarchical topology (1/2) Pyramid Hierarchical ring (Hierarchical 2 D mesh) ＮＣ論２ 24

Hierarchical topology (2/2) Cube-connected cycles RDT (Recursive Diagonal Torus) ＮＣ論２ 25

Hypermesh (spaninng-bus hypercube) Single or multiple buses ＮＣ論２ 26

Base-m n-cube (hyper-crossbar) 770 070 777 077 707 000 007 8 x 8 crossbar Base-8 3 -cube (Toshiba Prodigy) ＮＣ論２ 27

Diameter and degrees (1/2) 2 D　 mesh 2 D torus 3 D torus binary n-cube #node N Ｎ Ｎ Ｎ = 2 n Diameter ２√N √N √N log N degree ４ ４ ６ log N ＮＣ論２ ３ 28

Diameter and degrees (2/2) Base-m n -cube #node Binary tree ring Ｎ Ｎ 2 log N N/2 3 2 N = mn Ｎ = n 2 n Diameter logm N degree CCC logm N 3 n/2 3 ＮＣ論２ ３ 29