Analytics driven testing Anusha Mukkavilli Associate Consultant Nibhanupudi

Analytics driven testing Anusha Mukkavilli – Associate Consultant Nibhanupudi Suguna Associate Consultant Capgemini India Pvt. Ltd Logo of your organization

Abstract Digitalization is fundamentally changing the business climate. To be successful, organizations must develop and adopt agile business models that respond faster to changing market and economic conditions. To support this business shift, the software industry re invented itself to allow for fast paced software systems evolution. This evolution has been made possible by many innovative technologies, architectures and tools (such as Cloud infrastructures, Open platforms, Building Blocks Method, etc. ) combined with new practices and methodologies In this context, development cycles are increasingly shorter while testing resources often remain at best unchanged. As consequence, testing became a big challenge mainly for large and scale software systems. IT professionals seek for industrial ways to maintain the quality of their systems as whole while they continuously and independently evolve. 2

Analytics driven testing provides practical ways to industrialize a software testing approach based on software metrics which consists of gathering data from various metrics as inputs and then building a business intelligent system that shows the test gaps as outputs. The Goal of analytics driven approach is to • Determine a quantitative measure of test coverage, which is imperative to measure quality. • Find untested areas of the software application. • Identify redundant test cases that do not increase test coverage. 3

Analytics driven approach in QA Gives insights on the customer sentiments that help in customer centric QA Provides insights to start, stop, and prioritize testing Increases testing efficiency and predictability Reduces overall cost due to early defect detection Accelerates time to market Going beyond the traditional QA methodologies and taking an analytics based approach has become key factor in the next generation of QA. This helps in predicting the future failure looking at the past data and taking the proactive measures for future. 4

Introduction For many software testing firms to accelerate growth in competitive world it is imperative for them to transform their business models. This is not possible until and unless the companies reinvent themselves to allow for fast paced software systems evolution. This evolution has been made possible by many innovative technologies, architectures and tools combined with new practices and methodologies which drastically shorten their time to market 5

What is Analytics? Analytics is defined as the process of measuring, recording, tracking, and analyzing data to study real time patterns and usage of an application or a workflow. Some of the key players in this field are Google Analytics, Yahoo Analytics and many others. Analytics data gives you better performance insights and also helps you understand, prioritize and in decision making on what needs testing and optimizing first on your application/website 6

Challenges that complicate testing process Different expectation in terms of KPI reports, Test results, Audit reports, Test management reports, other metrics, etc. Unable to produce desired and on demand analytical reports Is the project going in the right track? Who is the right tester for this assignment? Where exactly I am incurring more cost? Will I be able to meet the deadline? What measures should I take to accomplish the project with in the set deadlines? 7

World Quality Report 2016 -17 8

Advantages of Analytics driven approach in Software Testing It helps in identifying right tester for particular task It helps you dis cover, mon i tor and set tar gets for your key suc cess met rics It helps in identifying issues impacting various areas of project It helps in proactively identifying the risks and mitigating the risks at the earliest stage It helps in identifying where the delay is and what is the issue It helps in Improve Planning, Quality and Delivery It Predictive Analytics helps in making right decisions at right time. 9

Gartner’s Hype Cycle for emerging trends in testing 2013 10

Analytics process 1. Analytics process components: A communications platform. A feedback system. Training. A text analytics engine. 2. Real time versus right time – information and insights must be provided at the right time and not necessarily continuously. 3. Insights – the process of interpretations must be standardized so that different managers come to the same conclusion after analyzing the data. 11

Analytics toolbox Descriptive analytics – creating simple counts, distributions, visualizations describing your data. Predictive analytics – predicting organizational and process behaviour, e. g. , can you predict at the beginning of a release how it will end? Prescriptive analytics – prescribe corrective actions and suggest mitigations after identifying aberrations, risks, etc. Way to develop and design a dashboard • Follow an accepted methodology • Specific process/goal in order to be useful. • Do not mix strategic metrics with operational metrics • Provide drill down capabilities to better focus on and target your issues. 12

Way to develop ‘meaningful’ metrics Goals need to be defined based on organizationally adopted quality frameworks (TMMI, ISO, IEEE, etc. ). Questions need to be defined so that their answers meet the goals. Defined metrics are a natural outcome of the questions. On which processes should organizations focus when considering the analytics initiative? The three major quality processes that directly impact the analytics initiative are: 1. Requirements management 2. Test automation 3. Defect management 13

How can we get organizational and management support for the QA analytics initiative? The organization defines a framework that drives the QA improvement function. Management understands that their success is directly proportional to the quality of the delivered applications. Management always needs to balance between committing resources to deliver quality applications and lowering QA costs. Budgets are justified by continuously demonstrating the quality of application delivery through efficient quality processes. 14

Business benefits-Real world example Tricentis Tosca a small Austrian startup raised $165 M to deliver continuous testing at the speed of Dev. Ops Tricentis being named a Leader in Gartner’s Magic Quadrant for Software Test Automation for the second year in a row. According to the Forrester report, “Tricentis Tosca provides top test automation and optimization design capabilities, test assets reuse, and combined automation. ” 15

Conclusion There are many heuristic approaches which are emerging to push the boundaries beyond pure analytics leading to continuous and touch less automation at the speed of thought. For example, MBT (model based testing) and Test Coverage, PBT (Theorem Proving Based Testing), AI, Machine Learning, Deep learning etc. The proposed approach proves to be not only effective in saving manual effort, time, and energy but also in helps in reducing the testing costs and deriving better ROI early in the testing life cycle. Ultimately, it improves efficiency and effectiveness of the testing operations. 16

References & Appendix https: //www. linkedin. com/pulse/analytics driven software testing hussein jibaoui http: //qa. siliconindia. com/qa expert/Analytics Driven Test Coverage Part I eid 211. html http: //www. softwaretestingnews. co. uk/ https: //www. capgemini. com/thought leadership/world quality report 2016 17 https: //www. tricentis. com/tricentis tosca testsuite/supported technology/ http: //www. gartner. com/newsroom/id/3412017 17

Author Biography Anusha Mukkavilli IT professional with 4+ years of work experience, currently working for Progressive Insurance account Nibhanupudi Suguna IT professional with 4+ years of work experience, currently working for Progressive Insurance account 18

Thank You!!! Logo of your organization 19