Developing a Cluster Strategy for NPACI All Hands

Developing a Cluster Strategy for NPACI All Hands Meeting Panel Feb 11, 2000 David E. Culler Computer Science Division University of California, Berkeley http: //www. cs. berkeley. edu/~culler 2/11/2000 NPACI Clusters

UCB Millennium Cluster of Clusters • x 86+Myrinet platforms w/ Gb. E inter-networking NTON Internet-2 Super. Net Ninja ½ TB PII Astro 8 x 2 Physics PIII 32 x 2 PIII-X 64 x 4 Gigabit Ethernet (Gb. E) PII 8 x 2 CE PII 8 x 2 NOW Bio PII 8 x 2 PIII DLIB PII Math 8 x 2 Mobile Svcs Kiosks • Distributed ownership, allocation, and management 2/11/2000 NPACI Clusters 2

Vineyard Cluster Architecture Applications / Services (ISPACE/Kiosks) PBS I/O MPI VEXEC TOOLS REXEC - VIA / GM, Gb. E - Multicast Mgmt / Monitoring • Distributed resource utilization and management in a “Vineyard” of Clusters. - NT / Linux (2. 2. x) - Stride Scheduler Rootstock Distribution 2/11/2000 NPACI Clusters 3

Clusters “own” HPC 2/11/2000 NPACI Clusters 4

Fundamental Advantages of Clusters • • Cost Performance / Cost Track leading edge of market technology Incremental scalability Availability Tremendous I/O performance Wide-Area Network performance – competitive internal network performance too • Allow specialization of networked services 2/11/2000 NPACI Clusters 5

Fundamental Challenges • Management • Complete system on every node – need scalable administration • Incremental scalability & availability => – heterogeneity – some parts inoperable at any time • The Cluster projects are making great progress in this area – eg: Millennium rootstock • Cluster tools are what you want for managing the desktops across your department 2/11/2000 NPACI Clusters 6

CS&E HPC hampered by “self-centered” usage model • Have my own application for my studies • Want the entire machine to myself • Want it now • Think “services” • Think “software” • The value is in your application. • Make it a service and make it available to the scientific community. • Put it on a cluster to deliver results 24 x 7 x 52 2/11/2000 NPACI Clusters 7

Example: TCAD Simulation Service • http: //cuervo. eecs. berkeley. edu/Volcano/ • star formation simulation • earthquake simulations • phylogeny, BLAST, . . . 2/11/2000 NPACI Clusters 8

Extreme Example • UCB Millennium / NOW has deliver 70 CPU years! • Simple special case, but. . . • Engineered for portability, adaptability, availability 2/11/2000 NPACI Clusters 9

What should NPACI do? To be relevant: • become a “Center of Expertise” for clusters • draw expertise toward the center for ease of dissemination • facilitate and encourage building clusters among the partners • invest in an interesting cluster “close to home” – cheap! Graft Millennium • invest in people to understand the implications To Lead: • Pioneer widespread computational science and engineering services • infiniband 2/11/2000 NPACI Clusters 10

from e-commerce to 2/11/2000 NPACI Clusters 11

Technical Backup Slides 2/11/2000 NPACI Clusters 12

Rootstock Mechanics 1. Cluster Stock - Rootstock build pages - Full Current Linux - all fixes and pckgs 3. CS power-on build Cluster System Distribution Center - xfer and localize DT - add local admin scripts - node build floppy K leased builds cluster stock - build - os - drvrs - mill SW - os mods Cluster cs IP network - SSL, SSH - Cluster Drivers - Cluster System Layers - rexec, mpe, pbs - Optional SW ($) - Cluster Kernal Mods 2/11/2000 CAN 2. Make the CS “graft” - specify IP address - pckg removes - dchp, dns, nis, . . . sanity check and build - resolv. conf, /etc/hosts, . . . constructs cluster build (lease) download CS build floppy . . . 4. Node power-on build - local stock from CS 5. Cluster Update button (future) - 2 nd dialtone, CF engine, rolling update NPACI Clusters 13

REXEC / VEXEC • Components – rexecd, rexec & vexecd Node A Node B Node C Node D rexecd Cluster IP Multicast Channel vexecd (Policy A) run indexer on Nodes AB at 3 credits/min vexecd (Policy B) “Nodes AB” minimum $ rexec %rexec –n 2 –r 3 indexer 2/11/2000 NPACI Clusters 14

Computational Economy • Market-based approach to resource allocation – Optimizes for user value Time Share A P I Batch Queue Economic F. E. Resources 2/11/2000 Resource Managers Access Modules Apps (Value) NPACI Clusters 15