Benchmarking Performance Evaluation Modeling and Prediction Erich Strohmaier

Benchmarking, Performance Evaluation, Modeling and Prediction Erich Strohmaier

ICL Benchmarking Activities • • • Linpack benchmark Park. Bench LLCbench The Performance Database Server TOP 500 Performance Analysis Tool for NAS

Linpack Benchmark • 3 main ‘flavors – 100 x 100 – 1000 x 1000 – Nx. N • R max from Nx. N used in the TOP 500

Park. Bench • Collection of – Low. Level benchmarks • sequential • MPI • PVM – Kernels • Linear Algebra • NAS PB Kernels – Applications • NAS PB (simulated applications) • PSTSWM

LLCbench • Low-Level Tests for various hardware aspects: • MPBench – Core MPI contructs • BLASBench – BLAS routines • Cache. Bench – In and out of cache data movement

Performance Database Server • Searchable Interface to DB of benchmark results such as: – Linpack – SPEC – (Dhrystone etc)

Basis for analysing the HCP market - Quantification of observations - Detection of trends • market, • architecture, • technology

TOP 500 Procedure • Listing of the 500 most powerful Computers in the World • Yardstick: Rmax/LINPACK • Updating halfyearly

TOP 500 list - Data shown • • • Manufacturer or vendor Computer Type indicated by manufacturer or vendor Inst. Site Customer Location and country Year of installation/last major update Field of Appl. Academic, Research, Industry, Vendor, Class. # Proc. Number of processors Rmax Maxmimal LINPACK performance achieved Rpeak Theoretical peak performance Nmax Problemsize for achieving Rmax N 1/2 Problemsize for achieving half of Rmax Nworld Position within the TOP 500 ranking

TOP 10

Performance Development

Manufacturer

Architectures

Performance Development

Performance Development

Performance Development

www. top 500. org.

NAS Performance Analysis Tool • To provide a tool to NAS users which allows – Access to performance data of NAS reference codes – Performing a variety of predefined performance tests on target systems in an easy and comparable ways – Analyzing user performance measurements using advanced statistical methods • Analyze scalability of his/her application • Compare its efficiency to other codes • Get a first prediction of attainable performance ranges on other NAS systems

Necessary Functionality • Broad variety of predefined performance tests • Simple interface do define custom performance tests • Reference measurement of NAS applications • Generation and maintenance of performance database

Necessary Functionality • Black Box performance analysis for individual codes • Cross-code and cross-platform performance analysis. • Tool for performance prediction for NAS applications • Detection and analysis of system and code performance signatures

System Architecture GUI: - Test definition database - All predefined performance test with performance models - Test suite setup file Create script - User data entry - Result display - Result analysis using various methods - Result commit to include in DB Benchmark Skeleton codes Test benchmarks Execute script Test results Performance DB Connected by same keys

GUI - Analysis Component Functions available for non-linar regression Input of measured data as: #proz time Data are NPB MG Class A on Steger Statistical output of analysis best model Data and regression display