An Automated approach towards DevOps based continuous Regression

An Automated approach towards Dev-Ops based continuous Regression Testing for SAP Landscapes Main Author - Navaneetha Kowdle (Associate Director) Co – Author - Akshay Mullick (Consultant) Cognizant 1

Abstract Ø Navigating the complex digital landscape demands an effective testing strategy. Ø Many of the world’s largest implementers of SAP are continuously looking to improve the effectiveness of an already mature QA organization. Ø Regression testing efficiency can be improved to a great degree using effective impact analysis tools. However, in the quest for more agility commercial tools often fall short. Ø Enterprises can significantly reduce test cycle time by using customized Testops solutions to achieve continuous regression testing. 2

Key Highlights Ø Comparative Study of Impact Analysis tools available in Market and why SAP’s impact analysis tool – BPCA (Business Process Change Analyzer) is recommended. Ø Solution Overview on how to use existing repositories, resources and tools to build automated solutions with minimal disruptions. Ø Insights into the best practices, implementation challenges and workarounds to successfully implement BPCA. Ø Description of how BPCA can be used to develop Test-Ops Solutions by implementing seamless end-to-end automated regression testing using a “Shift -Left” paradigm. Ø Real success stories on test effort saving of up to 70% for each regression cycle. 3

Erstwhile approach Vs Impact based approach Normal RT Scoping : Impact based RT Scoping : Regression Test Scope “Baseline” Optional Scenarios fragile (dependency on skills of individuals) Baseline Testing Driven by business critical scenarios covering core processes Understanding the changed Objects Limiting Test Scope “Gap Identification” will identify Gaps that our RT Capability misses and triggers test case creation of those before RT cycle

Benefits of Impact Analysis Targeted Testing Advancing Benefits Through Test-Ops Faster Time to Market Reduced Effort Reduced Cost Increased Resilience Early Defect Detection

Why BPCA (Impact Analysis Tool) for SAP Ø BPCA is a part of Sol. Man Suite and free to use. Ø Provided a means to make use of existing Test repository to build a blueprint, thus providing a 1 -1 mapping. Ø More granular than other tools compared, thus providing finer impact ( Subroutine/Form level). Ø Features such as configurable Test Scope Optimization / Multiplicity parameter to define extent of testing Ø Advantageous in building customized solution for continuous testing

Implementing BPCA based RT– Key Steps Connection Setup Business Blueprint § Set up of business process hierarchy § Business Requirements § TBOM Creation (Static/Semi. Dynamic/Dynamic) CHARM § § Change Document information Transport Information Business Process Change Analyzer § Impact Analysis § Test Scope Optimization § Multiplicity Parameter Test Planning &Setup § TAO/UFT Scripts Re-structuring § Requirements Mapping § Test Lab Creation Test Execution § Impacted Automated Test Scenario Execution § Impacted Manual Test Scenario Execution Defect Tracking § Defect raised and tracked for failed scenarios

Regression testing – Impact based approach BPCA & TSO Executed with Transports collected from CHARM BPCA & TSO Developer implements the change and release the transport S 7 D 4716 S 7 D 4717 S 7 D 4716 Change and Transport information flows to CHARM S 7 D 4713 S 7 D 4714 HP Quality Center S 7 D 4712 QA Environment S 7 D 4716 Pre-Prd Environment Automated Test Scenarios selected and executed in Pre-PRD Test Lab created of impacted Scenarios after TSO from BPCA

Pain Points encountered during BPCA implementation Ø Substantial initial effort for dynamic TBo. M creation. Ø OSS Notes for enabling TBo. M creation (auto and manual) – ended up as guinea pigs Ø TAO scenarios restructuring – PFA consolidation etc. Ø Need to review HP ALM structure in order to align with Blueprint / BPCA needs Ø Technical Connection Setup with HP ALM Ø Approach for auto TBo. M creation for UFT scenarios Ø Connections between SAP Sol. Man and Managed Systems

Taking it Further – Pioneering work Ø Increasing Test Coverage Ø Continuous RT – adaption of Shift Left approach using Dev. Ops “The V Model” “Shift Left” Integration testing ensures that the e 2 e process as described in the design documents function as described. User acceptance testing ensures that changes meet the business requirements. Gate 2 A Gate 1 User Acceptance Test Gate 3 Gate 2 B Business Requirements Light Regression Test Gate 0 Business Requirements Gate 0 Gate 3 Gate 2 B Regression testing ensures that a change or fix has not caused faults in unchanged parts of the system. Regressio n Test Gate 1 Functional Design Integration Test Cycle 1 / 2 Gate 2 A User Acceptance Test Functional Design Unit / Component / Assembly Test Integration Test Cycle 1 / 2 Technical Design H/H New or changed requirements are brought to production by projects. Fixes to incidents and problems are brought to production by application maintenance. …tu rnin Technical Design H/H g in to C ont inuo us R T… Build Unit / Component / Assembly Test …limits speed and flexibility Build Focus on testing „new“ functionality Focus on resilience …will enable Fast Track Deployments not compromising on resilience finally resulting into Continuous Deployment…

Enterprise Dev. Ops -Motivation and Guiding Principles Motivation Ø Speed-up of go-lives of Enhancements / Innovations Ø Cost reduction through higher degree of automation Ø Much earlier detection of defects from regression tests Guiding principles Ø Continuous development, regression testing and deployment. Ø Monthly releases – mid-term goal: 2 weeks. Ø Reduced System Landscape with max. 4 systems: DEV, QAS, Pre-PRD, PRD. Ø Agile development approach replacing traditional waterfall approach

Continuous Automated Regression Testing BPCA & TSO Automated Inbuilt Mechanism Automated Manual BPCA Process Execution S 7 D 4716 S 7 D 4717 Automated Manual Test Lab Process Creation Test Data Solution S 7 D 4716 S 7 D 4714 HP Quality Center S 7 D 4713 S 7 D 4712 QA Environment Automated Manual Test Process Execution Auto scenarios Automated Manual Test Process Selection

Benefits seen post implementation Test Coverage Accelerated Approach § Gaps identified which are not covered under Baseline Test Scenarios. § Shortened overall Release Cycle window from 4 weeks to 3 weeks. § Gap Fulfillment increased test coverage by 15% yearly. § Defects are caught earlier in Release Cycle. Cost Tangible Benefits § § Up to 60% reduction in testing effort. § Brought in 25% more agility which helps in faster “Time to Market”. § Save cost by engaging less testing resources due to targeted testing. Fast ROI within 2 -3 Monthly Release Cycles

References & Appendix 14

Author Biography Navneeth Kowdle is an Associate Director with the Process & Quality consulting division ( a part of Cognizant Business Consulting group). In this capacity , his role entails overall management of PQC practice in the Communication , Media and Technology space, in addition to personally getting engaged with customers in providing consultancy services in niche areas of Quality Assurance such as Dev. Ops, Business Process Testing , QA transformation strategy etc Main Author Akshay Mullick is working as an Consultant with the Process & Quality consulting division ( a part of Cognizant Business Consulting group). My role is to act as a bridge between business and technology for client and propose solutions which addresses their problem statement. My area of expertise is Agile, Dev-Ops, Change Management, Tools & Automation around Impact based testing Co - Author 15

Thank You!!! 16