Connection Machine Architecture Greg Faust Mike Gibson Sal

Connection Machine Architecture Greg Faust, Mike Gibson, Sal Valente CS-6354 Computer Architecture Fall 2009 1

Historic Timeline • • • 1981: MIT AI-Lab Technical Memo on CM 1982: Thinking Machines Inc. Founded 1985: Danny Hillis wins ACM “Best Ph. D” Award 1986: CM-1 Ships 1987: CM-2 Ships 1991: CM-5 Announced 1991: CM-5 Ships 1994: TMI Chapter 11 – Sun/Oracle pick bones Heavily DARPA funded/backed $16 M+ Direct Contracts plus subsidized CM sales 2

Involved Notables • • • Danny Hillis – CM inventor and TMI Founder Charles Leiserson – Fat tree inventor Richard Feynman – Noble Prize winning Physicist Marvin Minsky – MIT AI Lab “Visionary” Guy Steele – Common Lisp, Grace Hopper Award Stephen Wolfram – Mathematica inventor Doug Lenat – Mind/Body problem philosopher Greg Papadopoulos – MIT Media lab, Sun CTO various others 3

CM-1 and CM-2 Architecture • • • Original design goal to support neuron like simulations Up to 64 K single bit processors (actually 3 bits in and 2 out) 16 Processors/chip, 32 chips/PCB, 16 PCBs/cube, 8 cubes/hypercube Hypercube architecture – Each 16 -Proc chip a hyper-node Each proc has 4 K bits of bit addressable RAM – Distributed Physical Memory – Global Memory Addresses • • • Up to 4 front-end computers talk to sequencers via 4 x 4 crossbar “Sequencers” issue SIMD instructions over a Broadcast Network Bit procs communicate via 2 D local HW grid connections (“NEWS”) Bit procs communicate via hypercube network using MSG passing Lots of Twinkling Lights!! 4

CM-1 CM-2 Architecture 5

CM-1 and CM-2 Programming • ISA supports: – Bit-oriented operations – Arbitrary precision multi-bit scalar Ops using bit-serial implementation on bit procs – Full Multi-Dimensional Vector Ops • “Virtual Processor” idea similar to CUDA threads but they are statically allocated • OS and Programming Tools run on front-ends • *Lisp as the initial programming language • Later C* and CM-Fortran 6

CM-2 Improvements • • 1 Weitek IEEE FP coprocessor per 32 1 -bit procs Up to 256 K bits of memory per processor Added ECC to Memory Implemented the IO subsystem – Up to 80 GByte RAID array called “Data Vault” uses 39 Striped Disks and ECC, plus spare disks on standby – High Speed Graphics Output • En-route MSG combining in H-Cube router • New implementation of Multi-Dimensional NEWS on top of H-Cube (special addressing mode) 7

CM-1 Photo 8

CM-5 vs CM-1 and CM-2 Significant departure from CM-1 and CM-2 Targeted at more scientific and business applications More Commercial Off-The-Shelf components (“COTS”) Large Array of SPARC Processing Nodes – 1 -bit processors are abandoned • Abandoned “NEWS” Grid and Hyper-Cube Networks • Delivered 1024 node machine, with claims 16 K nodes possible • Even More Twinkling Lights! • • 9

CM-5 Photo – Watch it Blink 10

CM-5 Overall Architecture • "Coordinated Homogeneous Array of RISC Processors“ or “CHARM” • Asymmetric Co. Processors Model – Large Array of Processor Nodes – Small Collection of Control Nodes • 2 Separate scalable networks – One for data – One for control and synchronization • Still uses striped RAID for high disk Band. Width 11

Division of Labor • Processor Nodes can be assigned to a “Partition” • One Control Node per Partition • Control Node runs scalar code, then broadcasts parallel work to Processor Nodes • Processor Nodes receive a program, not an instruction stream, have own Program Counter • Processor nodes can access other node's memory by reading or writing a global memory address • Processor Nodes also communicate via MSG passing • Processor Nodes cannot issue system calls 12

Control Nodes • • • Full Sun Workstations Running UNIX Connected to the “Outside World” Handles Partition Time Sharing Connected to both data and control networks Performs System Diagnostics 13

Processor Nodes • Nodes are a 5 -chip microprocessor – Off the Shelf SPARC processor @ 40 MHz – 32 MBytes local node memory – Multi-port memory controller for added BW – “Caching techniques do not perform as well on large parallel machines” – Proprietary 4 -FPU Vector coprocessor – Proprietary network controller 14

CM-5 Processor Node Diagram 15

Data Network Architecture • Point to Point Inter-node communication and I/O • Implemented as a Fat Tree – Fat Trees invented by TMI employee Charles Leiserson • • Claim: Onsite Band. Width Expandable Delivering 5 GB/sec Bisection BW on 1024 node machine Data router chip is a 8 x 8 crossbar switch Faulty nodes are mapped out of network – Programs can not assume a network topology • Network can be flushed when Time Share swaps occur • Network, not processors, guarantee end to end delivery 16

Fat Tree Structure 17

Separate Control Network • • • Synchronization & control network Complete Binary Tree organization Provides broadcast capability Implements barrier operations Implements interrupts for timesharing Performs reduction operators (Sum, Max, AND, OR, Count, etc) 18

CM-5 Programming • Supports multiple Parallel High Level Languages and Programming Styles – Including Data Parallel Model from CM-1 and CM-2 • Goal: Hide many decisions from programmers – CM-1, CM-2 vs CM-5 ISA changes – Use of Processor Node CPU vs Vector Co. Processors – Partition Wide Synchronizations generate by Compiler • Is it MIMD, SPMD, SIMD? – “Globally Synchronized MIMD” 19

Sample CM Apps • Machine Learning – Neural Nets, concept clustering, genetic algorithms • • VLSI Design Geophysics (Oil Exploration), Plate Tectonics Particle Simulation Fluid Flow Simulation Computer Vision Computer Graphics , Animation Protein Sequence Matching Global Climate Model Simulation 20

References Danny Hillis Ph. D: The Connection Machine Inc: The Rise and Fall of Thinking Machines Wiki: Connection Machine ACM: The CM-5 Connection Machine ACM: The Network Architecture of the CM-5 IEEE: Architecture and Applications of the Connection Machine • IEEE: Fat-trees: universal networks for hardware-efficient supercomputing • Encyclopedia of Computer Science and Technology • • • 21