An Automated approach towards DevOps based continuous Regression

An Automated approach towards Dev-Ops based continuous Regression Testing for SAP Landscapes Main Author - Navaneetha Kowdle (Associate Director) Co – Author - Akshay Mullick (Consultant) Cognizant Technologies

Abstract Ø SAP implementations can be extremely complex and test repositories, built over a time, can be voluminous Ø The constant need for changes impact existing functionality continuously and QA in general & Regression testing in particular are key activities to ensure nothing is broken Ø Many of the world’s largest implementers of SAP are continuously looking to improve the effectiveness of an already mature QA organization. Ø Regression Testing, although a key activity, takes time and effort for each release and QA organizations are looking at making this effective and agile Ø Focused testing using an impact based approach can help reduce RT scope whilst maintaining efficacy Ø A customized Dev-Ops solution for Regression Testing SAP systems can dramatically reduce time and effort Ø This presentation presents a successful implementation of a dev-ops based continuous Regression Testing approach using Impact analysis 2

An Automated approach towards Dev-Ops based continuous Regression Testing for SAP Landscapes Covered in Two Phases ØPhase 1 - Impact Analysis - Tool Selection & implementation ØPhase 2 - Continuous RT – adaption of Shift Left approach using Dev. Ops 3

Phase 1 - Erstwhile approach Vs Impact based approach Normal RT Scoping : Impact based RT Scoping : Regression Test Scope “Baseline” Optional Scenarios fragile (dependency on skills of individuals) Understanding the changed objects Limiting Test Scope Baseline Testing Driven by business critical scenarios “Gap Identification” will identify that covering core processes our RT Capability misses certain test cases and triggers creation of those before RT cycle

Why BPCA (Impact Analysis Tool) for SAP Ø BPCA is a part of SAP-Sol. Man Suite Ø Provided a means to make use of existing Test repository to build a blueprint, thus providing a 1 -1 mapping. Ø BPCA is capable of understanding existing test assets’ coverage (TBo. Ms) Ø More granular than other tools compared, thus providing finer impact (Subroutine/Form level). Ø Features such as configurable Test Scope Optimization / Multiplicity parameter to define extent of testing Ø Advantageous in building customized solution for continuous testing

Implementing BPCA based RT– Key Steps Connection Setup Business Blueprint § Set up of business process hierarchy § Business Requirements § TBOM Creation (Static/Semi. Dynamic/Dynamic) CHARM § § Change Document information Transport Information Business Process Change Analyzer § Impact Analysis § Test Scope Optimization § Multiplicity Parameter Test Planning &Setup § TAO/UFT Scripts Re-structuring § Requirements Mapping § Test Lab Creation Test Execution § Impacted Automated Test Scenario Execution § Impacted Manual Test Scenario Execution Defect Tracking § Defect raised and tracked for failed scenarios

Pain Points encountered during BPCA implementation Ø Effort for TBo. M creation § Manual § Also TAO with Association, PFA Consolidation, link to HP ALM Requirements module Ø OSS Notes for enabling TBo. M creation (auto and manual) – guinea pig approach Ø TAO scenarios restructuring – PFA consolidation etc to Ø Need to review HP ALM hierarchical structure in order to align with Blueprint / BPCA needs Ø Technical Connection Setup with HP ALM Ø Approach for auto TBo. M creation for UFT scenarios Ø Connections between SAP Sol. Man and Managed Systems

Regression testing – Impact based approach BPCA & TSO Executed with Transports collected from CHARM BPCA & TSO Developer implements the change and release the transport S 7 D 4716 S 7 D 4717 S 7 D 4716 Change and Transport information flows to CHARM S 7 D 4713 S 7 D 4714 HP Quality Center S 7 D 4712 QA Environment S 7 D 4716 Pre-Prd Environment Automated Test Scenarios selected and executed in Pre-PRD Test Lab created of impacted Scenarios after TSO from BPCA

Data Speak – End of Phase 1

Phase 2 -Taking it Further – Pioneering work Ø Increasing Test Coverage Ø Continuous RT – adaption of Shift Left approach using Dev. Ops “The V Model” “Shift Left” Integration testing ensures that the e 2 e process as described in the design documents function as described. User acceptance testing ensures that changes meet the business requirements. Gate 2 A Gate 1 User Acceptance Test Gate 3 Gate 2 B Business Requirements Light Regression Test Gate 0 Business Requirements Gate 0 Gate 3 Gate 2 B Regression testing ensures that a change or fix has not caused faults in unchanged parts of the system. Regressio n Test Gate 1 Functional Design Integration Test Cycle 1 / 2 Gate 2 A User Acceptance Test Functional Design Unit / Component / Assembly Test Integration Test Cycle 1 / 2 Technical Design H/H New or changed requirements are brought to production by projects. Fixes to incidents and problems are brought to production by application maintenance. …tu rnin Technical Design H/H g in to C ont inuo us R T… Build Unit / Component / Assembly Test …limits speed and flexibility Build Focus on testing „new“ functionality Focus on resilience …will enable Fast Track Deployments not compromising on resilience finally resulting into Continuous Deployment…

Continuous Automated Regression Testing BPCA & TSO Automated Inbuilt Mechanism Automated Manual BPCA Process Execution S 7 D 4716 S 7 D 4717 Automated Manual Test Lab Process Creation Test Data Solution S 7 D 4716 S 7 D 4714 HP Quality Center S 7 D 4713 S 7 D 4712 QA Environment Automated Manual Test Process Execution Auto scenarios Automated Manual Test Process Selection

Benefits of Impact Analysis Targeted Testing Advancing Benefits Through Test-Ops Faster Time to Market Reduced Effort Reduced Cost Increased Resilience Early Defect Detection

Benefits seen post implementation Test Coverage Accelerated Approach Ø Gaps identified which are not covered under Baseline Test Scenarios. Ø Shortened overall Release Cycle window by approximately 3 working days Ø Gap Fulfillment increased test coverage by approximately 15% in year 1. Ø Defects are caught earlier in Release Cycle. Cost Ø Ø Ø Save cost by engaging less number of testing resources due to targeted testing. Reduced Regressing Cycle time saved Testing Cost Early defect detection/correction reduced debug/retest effort significantly Tangible Benefits Ø Up to 60% reduction in testing effort. Ø Brought in 25% more agility which helps in faster “Time to Market”.

Author Biography Navneeth Kowdle is an Associate Director with the Process & Quality consulting division ( a part of Cognizant Business Consulting group). In this capacity , his role entails overall management of PQC practice in the Communication , Media and Technology space, in addition to personally getting engaged with customers in providing consultancy services in niche areas of Quality Assurance such as Dev. Ops, Business Process Testing , QA transformation strategy etc Main Author Akshay Mullick is working as an Consultant with the Process & Quality consulting division ( a part of Cognizant Business Consulting group. His role is to act as a bridge between business and technology for client and propose solutions which addresses their problem statement. His area of expertise is Agile, Dev-Ops, Change Management, Tools & Automation around Impact based testing Co - Author 14

Thank You!!! Questions ? 15