Performance Testing Distributed Systems Concepts and Terminology v

Performance Testing - Concepts and Terminology The document structure The document describes performance testing in two parts: General performance concepts Performance testing concepts 2 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology The document structure MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 4. Performance characteristics General performance concepts 5. Measured objects 6. Measurement data objectives and attributes 7. Abstract Performance metrics 8. Performance data processing 9. General performance test concepts Performance testing concepts 10. Performance test environment 11. Performance test specification concepts 12. Workload concepts 3 © Copyright 2008, Soft. Well Performance AB

Part 1 General performance concepts 4 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Performance characteristics MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 4. Performance characteristics Categorizing performance characteristics Powerfulness characteristics Reliability characteristics Efficiency characteristics 5 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Measured objects MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 5. Measured objects Measured services Measured components Service concepts Service characteristics Service interfaces 6 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Measurement data objectives and attributes MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 6. Measurement data objectives and attributes Performance metric objectives Measurement data attribute sets Metric types Processing attributes Identification attributes Unit attributes Conditional attributes 7 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Abstract performance metrics MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 7. Abstract Performance metrics Abstract powerfulness metrics Abstract reliability metrics Abstract efficiency metrics 8 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Performance data processing MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 8. Performance data processing Steps in performance data processing Collection and storage of raw performance data Condensation and normalization of raw performance data Performance data computations Evaluation of performance data Presentation of performance data 9 © Copyright 2008, Soft. Well Performance AB

Part 2 Performance testing concepts 10 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology General performance test concepts MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 9. General performance test concepts Performance test phases Performance test objectives Performance objectives and performance requirements Performance measurement conditions Performance targets Performance measurement standards Some performance measurement characteristics 11 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Performance test environment MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 10. Performance test environment Test environment concepts System Under Test concepts Test System concepts 12 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Performance test specifications MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 11. Performance test specifications Elements of performance test specifications Test objectives Test conditions Test configurations Test data specifications Test evaluation specifications 13 © Copyright 2008, Soft. Well Performance AB

Performance Testing - Concepts and Terminology Workload concepts MTS – Performance Testing Distributed Systems 1 – Concepts and Terminology 12. Workload concepts Workload set or Traffic set Workload content Workload volume Load concepts Workload time distribution 14 © Copyright 2008, Soft. Well Performance AB

What is next? 15 © Copyright 2008, Soft. Well Performance AB

Performance Testing – Next step Proposal MTS – Performance Testing Distributed Systems 2 – Methodology and Realization The next document should describe • Methods in all phases of performance tests • How methods are applied The purpose is to • Describe methodology of performance testing in a generalized manner • Describe how methods are applied • • Start the groundwork on a framework for performance testing based on a common view of abstract performance metrics and characteristics 16 © Copyright 2008, Soft. Well Performance AB

MTS objectives for Performance Testing 17 © Copyright 2008, Soft. Well Performance AB

MTS objectives for Performance Testing The MTS objectives should be to: • Develop a formalized view of performance test methods • Develop a formalized view of performance tests That can be used as: • A framework for applied performance tests and test suites • A guideline for design of performance tests tools • A guideline for performance test standards 18 © Copyright 2008, Soft. Well Performance AB

Some long term goals on Performance and Performance Testing 19 © Copyright 2008, Soft.

Long term goals for Performance Testing Some examples Performance is an increasingly important aspect of a design or a system. Performance will be an increasingly important aspect of a standards. A validated function works under most circumstances in production, validated performance is not a bulletproof guarantee for production. A formalized view of performance and performance testing enables: • • A complete specification of a system Modeling and design of a system’s performance Simple monitoring of performance in design and production Far better ways for predicting performance in production Intelligent performance test tools that can be built into the system A continuously improved understanding of performance A safer investment in software development 20 © Copyright 2008, Soft. Well Performance AB