Benchmarking Performance Evaluation Modeling and Prediction Erich Strohmaier
Benchmarking, Performance Evaluation, Modeling and Prediction Erich Strohmaier
ICL Benchmarking Activities • • • Linpack benchmark Park. Bench LLCbench The Performance Database Server TOP 500 Performance Analysis Tool for NAS
Linpack Benchmark • 3 main ‘flavors – 100 x 100 – 1000 x 1000 – Nx. N • R max from Nx. N used in the TOP 500
Park. Bench • Collection of – Low. Level benchmarks • sequential • MPI • PVM – Kernels • Linear Algebra • NAS PB Kernels – Applications • NAS PB (simulated applications) • PSTSWM
LLCbench • Low-Level Tests for various hardware aspects: • MPBench – Core MPI contructs • BLASBench – BLAS routines • Cache. Bench – In and out of cache data movement
Performance Database Server • Searchable Interface to DB of benchmark results such as: – Linpack – SPEC – (Dhrystone etc)
Basis for analysing the HCP market - Quantification of observations - Detection of trends • market, • architecture, • technology
TOP 500 Procedure • Listing of the 500 most powerful Computers in the World • Yardstick: Rmax/LINPACK • Updating halfyearly
TOP 500 list - Data shown • • • Manufacturer or vendor Computer Type indicated by manufacturer or vendor Inst. Site Customer Location and country Year of installation/last major update Field of Appl. Academic, Research, Industry, Vendor, Class. # Proc. Number of processors Rmax Maxmimal LINPACK performance achieved Rpeak Theoretical peak performance Nmax Problemsize for achieving Rmax N 1/2 Problemsize for achieving half of Rmax Nworld Position within the TOP 500 ranking
TOP 10
Performance Development
Manufacturer
Architectures
Performance Development
Performance Development
Performance Development
www. top 500. org.
NAS Performance Analysis Tool • To provide a tool to NAS users which allows – Access to performance data of NAS reference codes – Performing a variety of predefined performance tests on target systems in an easy and comparable ways – Analyzing user performance measurements using advanced statistical methods • Analyze scalability of his/her application • Compare its efficiency to other codes • Get a first prediction of attainable performance ranges on other NAS systems
Necessary Functionality • Broad variety of predefined performance tests • Simple interface do define custom performance tests • Reference measurement of NAS applications • Generation and maintenance of performance database
Necessary Functionality • Black Box performance analysis for individual codes • Cross-code and cross-platform performance analysis. • Tool for performance prediction for NAS applications • Detection and analysis of system and code performance signatures
System Architecture GUI: - Test definition database - All predefined performance test with performance models - Test suite setup file Create script - User data entry - Result display - Result analysis using various methods - Result commit to include in DB Benchmark Skeleton codes Test benchmarks Execute script Test results Performance DB Connected by same keys
GUI - Analysis Component Functions available for non-linar regression Input of measured data as: #proz time Data are NPB MG Class A on Steger Statistical output of analysis best model Data and regression display
- Slides: 23
