Recap 1 Computational Power Improvement C P I

Recap 1

Computational Power Improvement C. P. I Multiprocessor Uniprocessor 1 2. . No. of Processors 2

Summing m Numbers Example: summing m numbers On a sequential computer we have, sum = a[0]; for (i=1; i<m; i++) { sum = sum + a[i]; } Would expect the running time be be roughly proportional to m. 3

Summing m Numbers in Parallel • What if we have N processors, with each calculating the m/N numbers assigned to it? • We must add these partial sums together to get the total sum. 4

Summing Multiprocessors Suppose we have N processors. P 1 P 1 1 2 3 P 2 P 2 1 2 3 The algorithm is as follows: 1. Each processor finds the local sum of its m/N numbers i P 3 P 3 1 2 3 j 2. Each processor passes its local sum to another processor in a coordinated way 3. The global sum is finally in processor P 11. 5

Shared Memory Multiprocessors MB MB P/C Cache NIC Bus/Custom-Designed Network Shared Memory 6

MPP (Massively Parallel Processing) Distributed Memory Multiprocessors MB : Memory Bus MB NIC : Network Interface Circuitry MB P/C LM LM NIC Custom-Designed Network 7

Cluster LD : Local Disk MB LD IOB : I/O Bus MB P/C M M Bridge IOB NIC IOB LD NIC Commodity Network (Ethernet, ATM, Myrinet) 8

Grid P/C IOC Hub/LAN SM P/C IOC Hub/LAN LD NIC SM SM LD NIC Internet 9

TOP 500 architectures 10