Software Quality Engineering Roadmap SE 450 Software Processes

  • Slides: 27
Download presentation
Software Quality Engineering Roadmap SE 450 Software Processes & Product Metrics 1

Software Quality Engineering Roadmap SE 450 Software Processes & Product Metrics 1

Business Objectives n Maximize Revenue n Minimize Costs SE 450 Software Processes & Product

Business Objectives n Maximize Revenue n Minimize Costs SE 450 Software Processes & Product Metrics 2

Maximize Revenue n n Improve market share: n Improve product quality n Increase customer

Maximize Revenue n n Improve market share: n Improve product quality n Increase customer satisfaction n Better marketing n Improve cycletime Increase product value: n Increase product quality n Define value proposition and match product capabilities and quality to customer needs SE 450 Software Processes & Product Metrics 3

Improve Market Share n n Improve product quality Increase customer satisfaction n Identify and

Improve Market Share n n Improve product quality Increase customer satisfaction n Identify and address key customer concerns n Continuous improvement process n Practices for measuring and improving satisfaction n Expectation management practices Better marketing n Match product to marketing needs n Improved requirements practices n Practices for measuring and improving marketing effectiveness Improve cycletime n Cycletime measurement and continuous improvement n Increase product value n Define value proposition and match product capabilities and quality to customer needs SE 450 Software Processes & Product Metrics 4

Minimize Costs – Improve Productivity n n n Productivity measurement and continuous improvement Practices

Minimize Costs – Improve Productivity n n n Productivity measurement and continuous improvement Practices for improving productivity: n Tools usage n Technology selection n Reuse and common architecture development n Streamlining and automation n Workflow, templates, tools n Task elimination Developer capability improvement through training SE 450 Software Processes & Product Metrics 5

Improve Project Management – Activity Management n Practices: planning, estimation, progress tracking n Metrics:

Improve Project Management – Activity Management n Practices: planning, estimation, progress tracking n Metrics: estimation accuracy, progress metrics n Estimation practices: n Maintain database of past projects n Estimate based on history of comparable projects n COCOMO (constructive cost model) provides questionnaire and formula for estimation based on organizational and project profile SE 450 Software Processes & Product Metrics 6

Improve Project Management – Risk Management n n Identify and track risks Metrics examples:

Improve Project Management – Risk Management n n Identify and track risks Metrics examples: n Percentage of unidentified high-impact risks among those that actually occurred n Percentage of unmitigated high-impact risks among those that actually occurred n Percentage of effort spent on risk management and mitigation SE 450 Software Processes & Product Metrics 7

Improve Project Management – People Management n Practices to measure and improve employee satisfaction

Improve Project Management – People Management n Practices to measure and improve employee satisfaction n Teaming practices and effectiveness: n Metrics: Turnover % SE 450 Software Processes & Product Metrics 8

Improve Project Management: In-Process Metrics n Slippage, progress n Inspection effectiveness (evolve as project

Improve Project Management: In-Process Metrics n Slippage, progress n Inspection effectiveness (evolve as project proceeds) n Defect rates n Test progress n Reliability growth curves SE 450 Software Processes & Product Metrics 9

Product Quality - Features n Requirements quality & effectiveness n Requirements prioritization & release

Product Quality - Features n Requirements quality & effectiveness n Requirements prioritization & release planning SE 450 Software Processes & Product Metrics 10

Product Quality - Performance n n n Defining performance objectives Practices: designing for performance,

Product Quality - Performance n n n Defining performance objectives Practices: designing for performance, performance analysis, performance testing Primarily measurements and not metrics SE 450 Software Processes & Product Metrics 11

Product Quality - Dependability n Reliability engineering: objective setting, practices and metrics n Availability:

Product Quality - Dependability n Reliability engineering: objective setting, practices and metrics n Availability: objective setting, practices and metrics n n Security: n Practices for identifying threats (defining objectives) n Security engineering practices n Security testing: threat injection Safety (similar to security and reliability) SE 450 Software Processes & Product Metrics 12

Product Quality - Usability n Specifying usability objectives n Usability engineering practices n n

Product Quality - Usability n Specifying usability objectives n Usability engineering practices n n Usability measurements: surveys, avg. learning time, keystrokes for frequent operations etc. Usability metrics: SUMI SE 450 Software Processes & Product Metrics 13

Product Quality - Evolvability n Evolvability goals n Practices for improving evolvability SE 450

Product Quality - Evolvability n Evolvability goals n Practices for improving evolvability SE 450 Software Processes & Product Metrics 14

Development Effectiveness n Requirements n Design n Testing & Inspections n Configuration Management n

Development Effectiveness n Requirements n Design n Testing & Inspections n Configuration Management n Quality Engineering Creative Research Systems SE 450 Software Processes & Product Metrics 15

Development Effectiveness Requirements n n Requirements effectiveness practices Elicitation practices, prototyping, requirements analysis practices,

Development Effectiveness Requirements n n Requirements effectiveness practices Elicitation practices, prototyping, requirements analysis practices, traceability n Requirements metrics n Activity metrics, also requirements defect injection rate SE 450 Software Processes & Product Metrics 16

Development Effectiveness - Design n n Practices for design effectiveness Modularity practices, design patterns

Development Effectiveness - Design n n Practices for design effectiveness Modularity practices, design patterns for achieving various quality attributes, design diagramming, design analysis techniques Design metrics Activity metrics, injection rates, product quality metrics, reuse percentage SE 450 Software Processes & Product Metrics 17

Development Effectiveness – Testing & Inspections n n n Practices for testing effectiveness Multistage

Development Effectiveness – Testing & Inspections n n n Practices for testing effectiveness Multistage testing, test strategy & planning, test case identification through equiv classes, automation of test generation, randomized parameters, automated testing Reliability engineering Test metrics DRE charts, defect rates and densities, reliability … Inspection practices and metrics SE 450 Software Processes & Product Metrics 18

Development Effectiveness – Configuration Management n Configuration management practices n Metrics for configuration management

Development Effectiveness – Configuration Management n Configuration management practices n Metrics for configuration management effectiveness n Configuration management defect injection rate SE 450 Software Processes & Product Metrics 19

Development Effectiveness – Quality Engineering Practices n Process Improvement / Continuous Improvement n Assessments

Development Effectiveness – Quality Engineering Practices n Process Improvement / Continuous Improvement n Assessments SE 450 Software Processes & Product Metrics 20

Quality Engineering Process n n n n n Initial process definition and tailoring Process

Quality Engineering Process n n n n n Initial process definition and tailoring Process compliance metrics (certifications, assessments) Process improvement Improve effectiveness of practices Improve breadth of process & practices (increased “maturity level”, CMMIstyle) Minimize volume of process, maximize effectiveness (agile development) Define set of processes that address the most frequent causes of defects and achieve more of the business objectives: maximizing cost-benefit ratio for process practices Requires deep understanding of process/quality relationships, high developer maturity and competence, and extensive customization to situational needs A stage of evolution beyond simply adding process breadth and doing improvement. SE 450 Software Processes & Product Metrics 21

Continuous Improvement n n n n Quality tools and causal analysis, defect elimination Identify

Continuous Improvement n n n n Quality tools and causal analysis, defect elimination Identify problem area Use fishbone diagrams to brainstorm and capture possible causes Use results to design data collection scheme and gather data about actual causes of problem - This is an extremely important step. The fishbone itself is just a set of guesses. It does not tell you which causes need addressing Use pareto charts to identify most significant causes of the problem Use defect elimination techniques (checklists, templates etc) to try to prevent the problem in the future Repeat continuously or until objective is achieved Effectiveness measured through trend lines for various process metrics SE 450 Software Processes & Product Metrics 22

Metrics Concepts n n Defining Metrics n GQM paradigm n Metrics must have significant

Metrics Concepts n n Defining Metrics n GQM paradigm n Metrics must have significant value: High impact (value provided to user of metric) compared to effort spent in collecting data. n Meaningfulness of metrics n Reliability n Validity n Importance of interpretation Gathering Measurements SE 450 Software Processes & Product Metrics 23

Goal-Question-Metric (GQM) Paradigm n n n What are the objectives for the metric? n

Goal-Question-Metric (GQM) Paradigm n n n What are the objectives for the metric? n Should identify the purpose of the metric – who wants to do what with it? n Includes “viewpoint”: whose viewpoint are we interested in for this? E. g. developer, project manager, organization manager, quality engineer, external assessor, customer. What specific questions do we want answered? n Identify specific aspects of the problem that we care about n Includes operational definitions of the quantities we want to measure Define measurements and metrics n What data will need to be gathered? Who will gather it and how? n What metric will be computed from the measurements? n Metrics must provide a basis for action n Metrics must have an interpretation: given a value, how will this be interpreted? n Metrics must have significant value: high impact (value provided to user of metric) compared to effort spent in collecting data n Meaningfulness of metrics n Reliability n Validity SE 450 Software Processes & Product Metrics 24

Importance of Interpretation n Understanding the intrinsic limitations of the metric n Understanding the

Importance of Interpretation n Understanding the intrinsic limitations of the metric n Understanding the limitations of the data collection process n Understanding situational factors that provide context to the numbers SE 450 Software Processes & Product Metrics 25

Gathering Measurements n n n Minimizing measurement effort: n Effort spent on providing data

Gathering Measurements n n n Minimizing measurement effort: n Effort spent on providing data is “wasted” effort from project perspective – needs to be minimized! n Often, person who has to put in effort is different from the one who gets immediate value! Higher the effort, less the motivation for the provider to supply good data Need for tools: n Tools improve accuracy of data, especially if collection is automated n May sometimes need to provide for manual adjustments to tool-generated data n Tools can automate processing of data and generate useful views (graphs) directly Motivating measurement: n Set high bar on value: only ask for measurements that provide considerable value n Make value visible to people providing measurements (clear how the gathered data will be used) n Try to provide value to those providing measurements: n e. g. customer concerns actually get addressed n e. g. developers get useful feedback that helps them avoid problems and reduce effort SE 450 Software Processes & Product Metrics 26

Metrics Perspective n Metrics are a small part of the overall quality engineering process.

Metrics Perspective n Metrics are a small part of the overall quality engineering process. n Define objectives for metrics. n Metrics interpretation is key. n Actions and improvements based on metrics interpretation is the primary value achieved in the use of metrics. SE 450 Software Processes & Product Metrics 27