Performance n Chapter 2 PH Introduction n n

Performance n Chapter 2 P&H

Introduction n n How does one measure report and sumarise performance? Complexity of modern systems make it very more difficult to access performance See through marketing hype Need to understand performance at multiple levels n Understand why a program performs poorly on a particular processor

c. f. Comercial Airplanes n Airplane Which of the following planes has the best performance? Passengers Boeing 737 -100 Boeing 747 BAC/Sud Concorde Douglas DC-8 -50 n n 101 470 132 146 Range (mi) Speed (mph) Throughput 630 4150 4000 8720 598 610 1350 544 Which of the planes is the fastest? Can also define computer performance in several ways 60, 398 286, 700 178, 200 79, 424

Performance n Purchasing perspective n given a collection of machines, which has the n n Design perspective n faced with design options, which has the n n best performance improvement ? least cost ? best performance / cost ? Both require n n n best performance ? least cost ? best performance / cost ? basis for comparison metric for evaluation Our goal is to understand cost & performance implications of architectural choices

Two notions of “performance” Plane DC to Paris Speed Passengers Throughput (pmph) Boeing 747 6. 5 hours 610 mph 470 286, 700 BAD/Sud Concorde 3 hours 1350 mph 132 178, 200 Which has higher performance? ° Time to do the task (Execution Time) – execution time, response time, latency ° Tasks per day, hour, week, sec, ns. . . (Performance) – throughput, bandwidth Response time and throughput often are in opposition

Definitions n Performance is in units of things-per-second n n bigger is better If we are primarily concerned with response time n performance(x) = execution_time(x) 1 " X is n times faster than Y" means Performance(X) n = ----------Performance(Y)

Example • Time of Concorde vs. Boeing 747? • Concord is 1350 mph / 610 mph = 2. 2 times faster = 6. 5 hours / 3 hours • Throughput of Concorde vs. Boeing 747 ? • Concord is 178, 200 pmph / 286, 700 pmph = 0. 62 “times faster” • Boeing is 286, 700 pmph / 178, 200 pmph = 1. 6 “times faster” • Boeing is 1. 6 times (“ 60%”)faster in terms of throughput • Concord is 2. 2 times (“ 120%”) faster in terms of flying time We will focus primarily on execution time for a single job

Basis of Evaluation Cons Pros • representative • portable • widely used • improvements useful in reality • easy to run, early in design cycle • identify peak capability and potential bottlenecks Actual Target Workload Full Application Benchmarks Small “Kernel” Benchmarks Micro benchmarks • very specific • non-portable • difficult to run, or measure • hard to identify cause • less representative • easy to “fool” • “peak” may be a long way from application performance

Benchmark Problems

SPEC 95 Eighteen application benchmarks (with inputs) reflecting a technical computing workload n Eight integer n n n Ten floating-point intensive n n go, m 88 ksim, gcc, compress, li, ijpeg, perl, vortex tomcatv, swim, su 2 cor, hydro 2 d, mgrid, applu, turb 3 d, apsi, fppp, wave 5 Must run with standard compiler flags n eliminate special undocumented incantations that may not even generate working code for real programs

Measuring Performance n Execution Time can be measured in several ways: n Elapsed Time n n n CPU time n n n counts everything (disk and memory accesses, I/O , etc. ) a useful number, but often not good for comparison purposes doesn't count I/O or time spent running other programs can be broken up into system time, and user time Our focus: user CPU time n time spent executing the lines of code that are "in" our program

Measuring Performance n n On Linux can use time command E. g. > time tar cvzf pam 2001. tgz pam 2001 real 0 m 19. 348 s user 0 m 0. 930 s sys 0 m 0. 660 s > time tar cvzf pam 2001. tgz pam 2001 real 0 m 1. 482 s user 0 m 0. 950 s sys 0 m 0. 230 s