Parallel Computers Demand for Computational Speed Continual demand

Parallel Computers

Demand for Computational Speed • Continual demand for greater computational speed from a computer system than is currently possible • Areas requiring great computational speed include numerical modeling and simulation of scientific and engineering problems. • Computations must be completed within a “reasonable” time period.

Grand Challenge Problems • A grand challenge problem is one that cannot be solved in a reasonable amount of time with today’s computers. Obviously, an execution time of 10 years is always unreasonable. Examples • Modeling large DNA structures • Global weather forecasting • Modeling motion of astronomical bodies.

Weather Forcasting • Atmosphere modeled by dividing it into 3 dimensional cells. Calculations of each cell repeated many times to model passage of time.

Global Weather Forecasting Example • Whole global atmosphere divided into cells of size 1 mile X 1 mile to a height of 10 miles (10 cells high) - about 5 X 108 cells. • Suppose each calculation requires 200 floating point operations. In one time step, 1011 floating point operations necessary. • To forecast the weather over 7 days using 1 -minute intervals, a computer operating at 1 Gflops (109 floating point operations/s) would take 106 seconds or over 10 days. • To perform the calculation in 5 minutes would require a computer oprating at 3. 4 Tflops (3. 4 X 1012 floating point operations/sec).

Modeling Motion of Astronomical Bodies • Each body attracted to each other body by gravitational forces. • Movement of each body predicted by calculating total force on each body. With N bodies, N - 1 forces to calculate for each body, or approx. N 2 calculations. (Nlog 2 N for an efficient approx. algorithm. ) • After determining new positions of bodies, calculations repeated. • A galaxy might have, say, 1011 stars. Even if each calculation could be done in 1 ms (an extremely optimistic figure), it would take 109 years for one iteration using the N 2 algorithm and almost a year for one iteration using an efficient Nlog 2 N approximate algorithm.

Parallel Computing • Using more than one computer, or a computer with more than one processor, to solve a problem. Motives • Usually faster computation - very simple idea that n computers operating simultaneously can achieve the result n times faster – it will not be n times faster for various reasons. • Other motives include: fault tolerance, larger amount of memory available, . . .

Speed Factors Notations • p = number of processors or processes • n = number of data items (used later)

Speed Factors

Amdahl's Law

Amdahl’s Law

Gustafson’s Law • Scaled speedup factor • The parallel time is constant (time constraint scaling) • Sp=f+(1 -f)p • Two assumptions: (1) Parallel time is constant (2) sequential part is constant, and not a function of p. • Amdahl’s law assumption: constant problem size scaling

Amdahl’s Law and Gustafson's Law • Amdahl's law approximately suggests: “Suppose a car is traveling between two cities 60 miles apart, and has already spent one hour traveling half the distance at 30 mph. No matter how fast you drive the last half, it is impossible to achieve 90 mph average before reaching the second city. Since it has already taken you 1 hour and you only have a distance of 60 miles total; going infinitely fast you would only achieve 60 mph. ” • Gustafson's law approximately states: “Suppose a car has already been traveling for some time at less than 90 mph. Given enough time and distance to travel, the car's average speed can always eventually reach 90 mph, no matter how long or how slowly it has already traveled. For example, if the car spent one hour at 30 mph, it could achieve this by driving at 120 mph for two additional hours, or at 150 mph for an hour, and so on.