Optimize Your SQA Practice Using a Full Lifecycle

Optimize Your SQA Practice Using a Full Lifecycle Approach Apply SQA across critical development process steps to ensure quality all the way to product delivery. Info-Tech Research Group, Inc. is a global leader in providing IT research and advice. Info-Tech’s products and services combine actionable insight and relevant advice with ready-to-use tools and templates that cover the full spectrum of IT concerns. © 1997 -2016 Info-Tech Research Group Inc. Info-Tech Research Group 1

Our understanding of the problem This Research is Is Designed For: This Research Will Help You: üSoftware quality assurance (SQA) managers üDevelop an SQA program that is measurable or related roles who are responsible for improving development process quality using an SQA initiative. üSQA managers or related roles who are having difficulty identifying where improvements need to be made to SQA. and includes program initiatives that are testable, maintainable, and traceable. üBuild a comprehensive SQA plan that outlines the appropriate tasks needed to inject quality all along the development process. This Research Will Also Assist: This Research Will Help You: Them: üBusiness stakeholders in understanding how üUnderstand where improvements need to be quality improvement can be achieved through the optimization of the development process. made to the SQA program, which align to an improved product delivery process. üRealize how improving process-oriented optimization initiatives can lead to improved product quality. Info-Tech Research Group 2

Executive summary Situation Info-Tech Insight • You are the individual responsible for SQA, and you have an • Your product is only as good as your • underperforming software quality assurance program that neglects process oversight of the entire development lifecycle. You are finding it hard to determine a causation between SQA process improvements and an improvement in product quality. Complication • Your SQA program is using the wrong set of metrics to measure how • process improvements influence product quality improvements. Roles and responsibilities alongside quality assurance initiatives are not well defined and allocated to individuals that are in a position to be held responsible for quality-related issues. • process. A robust development and SQA process produces artifacts that are highly testable, easily maintained, and strongly traceable across the development lifecycle, ensuring that the product delivered meets expectations set out by the business. A small issue within your development process can have a ripple effect on the level of quality in your products. Understand what you don’t know and identify areas within your SQA practice that require attention. Resolution • Optimize your current SQA program by adopting quality assurance tasks that achieve process-product alignment. • • Implement initiatives that are highly testable, easily maintained, and strongly traceable across the development lifecycle. Create a comprehensive SQA plan that delineates roles and responsibilities as they relate to quality assurance. Ensure tasks and procedures improve process efficiency and quality, and formalize metrics that help to implement a continuous improvement cycle for SQA. Develop an optimization roadmap that outlines the major initiatives needed to improve the SQA process. Info-Tech Research Group 3

ANALYST PERSPECTIVE Quality over quantity. Can your software quality assurance program attest to this? While quality is seen as something one can freely achieve, having a lack of mitigation steps when your product goes on to lose quality has major repercussions on how it is perceived and its ability to meet technical requirements. One small quality issue found anywhere along the development process could result in the delivery of a failed product. It’s really about deploying the right quality assurance initiatives, and the timing of when they are executed. And when it comes to the “right” initiatives, think of tasks that enforce or produce software artifacts that are testable, maintainable, and traceable all along the development process. These three attributes help to impose conditions on how artifacts are created – while it pushes your teams to execute against a standard, it also enables your teams to achieve quality effortlessly given that the right metrics are in place. Kevin Kim Research Manager, Application Development Practice Info-Tech Research Group 4

Schools of thought for SQA WHAT IS SOFTWARE QUALITY ASSURANCE (SQA)? SQA is defined as: “(1) A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements…” (IEEE. 1990. IEEE Standard Glossary of Software Engineering Terminology. Institute of Electrical and Electronics Engineers. IEEE 610. 12 -1990). What industry standard approaches for SQA does Info-Tech incorporate? ITIL emphasizes through its framework the achievement of expected product quality through the implementation of strong quality assurance processes. ITIL CMMI heavily emphasizes maturing an organization’s quality assurance processes through various levels. Each maturity level builds upon the previously developed SQA processes. Info-Tech takes a process-centric and measured approach to SQA as each software artifact created throughout the development lifecycle is to be highly testable, easily maintained, and strongly traceable. Info-Tech Research Group 5

Organizations are facing three main challenges when implementing SQA Without the correct understanding of what SQA is intended for and the proper checks and balances in place, it is difficult to know that the product you provide is exactly what was specified in the original requirements. The three common SQA challenges facing organizations today: 1 POOR ALIGNMENT The business does not buy into QArelated roles and responsibilities, which reduces the motivation to conduct QA initiatives at various stages of the SDLC. Without early QA involvement in the development process, opportunities to improve requirements and ensure testability are missed. 2 POOR GOVERNANCE QA teams are often positioned as an advisory board more than decisionmaking mechanism for development. Suggestions that come from a QA team are often overlooked and overruled. Lack of process standardization is biased toward developers where development activities are conducted as they see fit without emphasis on overall quality or impact on other teams. 3 LACK OF SKILLS Development teams are not appropriately trained to perform certain core tasks that are required by their functions, which leads to poor quality. QA teams are not properly equipped to measure the effectiveness of their QA program, which inhibits their ability to adapt QA initiatives to improve quality teams. Source: LMO The organizational barriers described above lead to common misconceptions of SQA: Software testing is the only measure for quality when the product is released into production. SQA is thought to be the practice solely of reducing bugs and defects after development. The responsibility of quality assurance rests on the shoulders of one or a few individuals. Source: Technology Management Info-Tech Research Group 6

Existing SQA processes are not capturing the results that you want to achieve Quality issues are systemic and cut across the software development lifecycle (SDLC). 75% of our surveyed clients state that IT quality management, which includes SQA, is a highly important process; however, only 15% say they are effective at it. Adding to this issue is the fact that more than 25% of organizations are challenged when implementing changes to established development processes. Source: Info-Tech Research Group, 2015 COMMON CAUSES OF DEPLOYING INAPT SQA PRACTICES 27% of organizations have 31% inflexible job roles or organizational structures. 5 th of organizations lack the appropriate budget to support SQA initiatives. 27% of organizations lack processes particularly around feature prioritization for development projects. most difficult challenge around testing user experience is identifying end-user expectations and requirements. Without the appropriate checks and balances in place, it is difficult to measure what you want to achieve through SQA. Source: Capgemini, 2015 Ensuring that the right processes are in place enables your teams to measure effectively and leverage the correct metric data to make informed decisions on where to improve. Info-Tech Research Group 7

Misalignment between SDLC teams leads to poor quality products Without the appropriate communication and collaboration, teams within your development projects aren’t informed of what quality assessments are used and where they are used. Team members with little to no sense of responsibility for quality assurance risk your development projects misaligning with their requirements. Poor interaction leads to poorly created software artifacts Teams deviate from inter-team collaboration and SQA processes when they are under pressure to meet deadlines, making it difficult to cultivate relationships between development teams. This type of activity leads them to create noncompliant software artifacts, which are then passed on to the next team in the development lifecycle who deem it unusable for what they need to do. Factors that influence misalignment between development teams Lack of communication Without the proper knowledge sharing and collaboration mechanisms in place, developers are prone to conduct testing in accordance with the code that they developed and not with the overall specifications that need to be met. Poor collaboration Poor execution of core development tasks increases the likelihood of defects, ultimately leading to the failure of the project. Process noncompliance When deadlines close in, developers and testers are prone to cut corners and often struggle to comply with standardized processes. Process deviation leads to a lack of transparency and misinforming other parties of changes made. Source: Mbekela and Brown As a result, only 29% of all software development projects are completed on time and on budget with satisfactory results (Standish Group, 2015). Info-Tech Research Group 8

Poor quality is costing you more than you think The cost of quality isn’t the cost of delivering quality, but rather the cost of not delivering it. Ensure you prevent quality issues before they occur. Mediating issues within development later rather than sooner can be detrimental in the long term. Leaving issues unresolved can be a costly exercise, increasing the cost associated with fixing the issue. Cost of fixing errors increases exponentially as your software progresses through the SDLC Relative Cost to Fix Issue While there are increasing pressures from the business to deliver software more quickly, emphasis on adhering to quality assurance processes is even more critical. Don’t let your issues go unresolved, or you will suffer the cost consequences during reactive defect and incident management. 0 Design & Architecture 1 X Post-Product Release 30 X Implementation 5 X Source: IBM, 2008 Quality is free. It’s not a gift, but it’s free. The ‘un-quality’ things are what cost money. – Philip B. Crosby Info-Tech Research Group 9

Do not limit your knowledge of QA to testing – QA needs to happen at all stages of the software lifecycle An SQA program that does not cover all areas of the SDLC is not an SQA program. You need accountability across the entire SDLC for quality to emerge. Understanding the SQA Landscape • • • Naturally, organizations begin quality assurance at the development and testing levels. As your products become technically complex and a greater number of resources are added to projects, instilling QA practices beyond development and testing is necessary for ensuring that high quality products are produced. QA is not about finding defects – it’s about preventing them before they happen. While testers and developers are positioned to detect defects and bugs, the main objective of QA is inherently different. QA is an umbrella term that spans across the entire SDLC, encompassing defect prevention techniques at all lifecycle stages. QA emphasizes the evaluation of process quality more than product quality. By ensuring that the process in place for development is of high quality, this guides your organization to create products that follow quality process standards and thus are high quality products. SQA Quality Planning SQA Process Execution Measurement & Analysis Idea Proposal Requirements Gathering Code Feature Customer Value Deploy Feature Test Feature Source: Tech. Target Info-Tech Research Group 10

Thought Model: Create a set of testable, maintainable, and traceable SQA practices Implement SQA practices that are measurable, align with your product quality goals and objectives, and are assigned appropriately to your teams. PROCESS-PRODUCT ALIGNMENT It is crucial to align business expectations with what is occurring during the development process. This can be achieved by ensuring your SQA practices enable your teams to generate software artifacts that are highly testable, easily maintained, and strongly traceable. Process. Product Alignment SQA Defined Roles & Responsibilities Process Measurability PROCESS MEASURABILITY DEPLOYING THE RIGHT METRICS WITH TOLERANCE LEVELS ALONGSIDE PROCESSES Measurability helps your practice become highly objective and enables your teams to mature from defect detection to prevention. This can be achieved by establishing tolerance bands for each metric used. DEFINED ROLES & RESPONSIBILITIES Set expectations of who does what by ensuring that processes are correctly executed by the right personnel. Setting these clear lines for responsibilities helps improve both inter-team and business -IT communication and alleviates any pain that previously surrounded work capacity concerns. Info-Tech Research Group 11

Use Info-Tech’s framework to develop robust SQA practices that can be used across projects Understand where your inputs and outputs lie to create a robust set of SQA practices for your development teams to follow. Input arrows represent the components that come into play in the SQA program refining process. This blueprint will take you through a three-phase approach to help you build a fully-functional SQA program that spans across your development process. Output arrows represent the value delivered and the benefits realized when optimizing your SQA program. Your program will in turn realize three defining elements: 1. Process-product alignment (as per thought model) 2. Process measurability 3. Defined roles and responsibilities 1. Assess Current SQA Capabilities Process-Product Alignment 2. Define SQA Target State Processes SQA Defined Roles & Responsibilities Process Measurability 3. Roadmap for Optimization Info-Tech Research Group 12

Support management with a fully-functional SQA program to improve decision making around development quality While it is management’s number one strategic priority to protect the corporate image in the context of SQA, this priority must be sustained through the implementation of a mature SQA program. Leverage Info-Tech’s blueprints that take you through the progression from quality assurance to implementing a fit-for-purpose quality management program. Info-Tech’s Program for Improving Quality Assurance and Management Maturity 3. Drive Efficiency and Agility with a Fit-for-Purpose Quality Management Program 2. Optimize Your SQA Practice Using a Full Lifecycle Approach 1. Develop a Mature Testing Practice to Produce Better Quality Applications Take your SQA program to the next level – develop a quality management Co. E that spans quality assurance practices across all business functions. Mature toward an SQA program by establishing a well-defined testing program. Time Info-Tech Research Group 13

Improve your SQA program to better meet business demands PROJECT STRUCTURE This blueprint will guide you through the steps to develop a holistic SQA plan and rollout strategy for implementing SQA optimization initiatives. Assess Your Current SQA Capabilities Define SQA Target State Processes Determine Your Optimization Initiatives for Improving Your SQA Practice Understand Your Current SQA Landscape Outputs Identify Problem Areas to Focus on Determine SQA Optimization Initiatives • • • High-Level Development Process SIPOC-MC RACI Document Your Current SDLC Process Flows Outputs • • • Outputs • Focus for addressing problem areas within SQA Document Target State SQA Process Outputs Process Flow Diagrams Control Points Metrics • • SQA Plan Completed SQA Assessment Workbook • • • Initiative Prioritization Risk/Benefit Assessment Effort Assessment Develop Rollout Strategy Outputs • • Roadmap Communication Plan Document Your Current Challenges Outputs • Current SQA Pain Points Info-Tech Research Group 14

Optimize Your SQA Practice Using a Full Lifecycle Approach – project overview 1. Assess Your Current SQA Capabilities 2. Define SQA Target State Processes 1. 1 Conduct a high-level assessment of where to focus your current state analysis 2. 1 Establish SQA metrics and tolerance limits across your SDLC 3. 1 Determine your optimization initiatives for improving your SQA process 1. 2 Document your high-level development process 2. 2 - 2. 5 Determine your target state for SQA processes within each stage of the SDLC (define, design, develop, test, and deploy/release) 3. 2 Gain the full scope of effort required to implement your SQA optimization initiatives 1. 3 Create a RACI chart to understand roles and responsibilities 1. 4 Undergo a SIPOC-MC analysis for problem areas identified in your SDLC 3. 3 Identify the enablers and blockers of your SQA optimization 3. 4 Define your SQA optimization roadmap 1. 5 Identify the individual control points involved with passing software artifacts through SDLC stages being assessed Best-Practice Toolkit 3. Determine Your Optimization Initiatives 1. 6 Identify problem areas within your SDLC as they relate to SQA Assess Your Current SQA Capabilities Define SQA Target State Processes Determine Your Optimization Initiatives Guided Implementations Module 1: Assess Your Current Capabilities Module 2: Define SQA Target State Processes Module 3: Determine Your Optimization Initiatives for Improving Your SQA Practice Phase 1 Outcome: Current State Assessment of SQA Phase 2 Outcome: SQA Plan Phase 3 Outcome: • Potential Optimization Initiatives for Improving SQA • SQA Optimization Roadmap Onsite Workshop Info-Tech Research Group 15

Info-Tech offers various levels of support to best suit your needs DIY Toolkit “Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful. ” Guided Implementation Workshop Consulting “Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track. ” “We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place. ” “Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project. ” Diagnostics and consistent frameworks used throughout all four options Info-Tech Research Group 16