SPEC OMP Benchmark Suite H Saito G Gaertner

SPEC OMP Benchmark Suite H. Saito, G. Gaertner, W. Jones, R. Eigenmann, H. Iwashita, R. Lieberman, M. van Waveren, and B. Whitney SPEC High-Performance Group

Structure of talk • Description of SPEC OMP Benchmarks • Scalability of SPEC OMP Benchmarks • Conclusion

SPEC OMP • Benchmark suite developed by SPEC HPG (High Performance Group) • Benchmark suite for performance testing of shared memory processor systems • Uses Open. MP versions of SPEC CPU 2000 benchmarks and candidates

Why Did SPEC Choose Open. MP? • Benchmark suite is focused on SMP systems • Open. MP is a standard, and is applicable to Fortran, C, and C++. • Directive based Open. MP allows serial version to remain largely intact. • Quickest path to parallel code conversion.

OMP/CPU 2000 Differences • Larger working set sizes, 1. 6 GB for OMPM 2001, 6. 5 GB for OMPL 2001; it is 200 MB for CPU 2000 • Longer run times (>1000 s/cpu for CPU 2000 vs >10, 000 s/cpu for OMP 2000 medium) • Focus on SMP systems, and issued by HPG • SPEC OMP based on work for CPU 2000,

OMP/CPU 2000 Similarities • Same tools used to run the benchmarks • Similar run and reporting rules • Uses geometric mean to calculate overall performance relative to a baseline system • Similar output format

OMP vs CPU 2000

SPEC OMP Benchmark Principles • Source code based • Limited code and directive modifications • Focused on SMP performance • Requires a base run • with no source modifications • single set of compiler flags for all benchmarks • SPEC supplied tools required to run benchmark

OMPM 2001 Benchmarks

SPEC OMP Benchmark Reference Runtimes

Program Memory Footprints

Benchmarks with good scaling

Benchmarks with good scaling up to 64 CPUs

Benchmarks with superlinear scaling

Benchmarks with poor scaling

Conclusion • SPEC OMP is a realistic set of benchmarks for SMP systems. • Up to date results are available on the SPEC web site at http: //www. spec. org/hpg/omp 2001 • Parallelization using Open. MP can be done relatively quickly, and on complex sections. • Scalability up to 128 CPUs is possible using Open. MP.

SPEC OMP is Here • Purchase through SPEC • Academic discount available • Membership encouraged