Challenges Faced by QA Team in Dev Ops

Challenges Faced by QA Team in Dev. Ops Testing Shanthala Bhat & Manager Manjunath A Padmanabha & Senior Test Lead Pw. C 1

Abstract The Dev. Ops is a concept for software development and delivery process that highlights collaboration between software development and operation professionals. The popularity of Dev. Ops has grown in recent years as it helps in releasing a software rapidly, more frequently and reliably. The Dev. Ops practice is being more often adopted for Microservice based architecture application as each service is small, easily tested and deployable through continuous integration. As per the survey published in January 2017 by Right. Scale, Dev. Ops adoption has increased from 66% in 2015 to 74% in 2016. As we move on its important to know the different challenges faced while implementing this concept from testing point of view. Hence this whitepaper focuses on providing information on different challenges faced while testing Microservice architecture based application implemented with Dev. Ops concepts. The testing community and test architects would find this paper very handy. 2

Business Case This business case is related to one of the world leader in financial technology solutions which provides payment solutions for credit cards, debit cards, electronic payments, and check-related services. Their current system was outdated, not scalable and increased with operational expenses. The team was looking for setting up new system fulfilling following goals of business expansion. • • • Faster time to market Allow easy, agile expansion for other countries, and for future acquisitions Reusable services supporting other business workloads Better deployment frequency Improved end user satisfaction Increased ability to build the right product by fast experimentation 3

Business Solution In order to achieve the above goals, the team decided to go with a concept of agile methodology, Dev. Ops, Microservice architecture style and BDD way. The architecture goal was to build an application enabled through use of cloud native capabilities, Microservices and Continuous Delivery. The application was comprised of a number of Microservices to avoid the problems of traditional monolithic applications as they are modelled around business domains. This implementation achieved the desired goals around availability and maintainability, Microservices exhibited below traits: • discoverable using a service registry • software load balancers and circuit breakers to protected from external dependency failure be independently deployable with provision for zero downtime • auto scaled up to meet demand to scale back when demand recedes to optimize cost • Be exposed as a RESTful API over HTTPS • The communication between each Microservice happened either through HTTPs between services or Message Queuing between components 4

Testing Approach for Business Case 5

Quality Engineering Approach 6

Challenges Faced - Generic Cultural Shift Skilled Resources Testing Mindset CI Readiness Code Quality Training SME (Dev. Test) Exploratory Tests Framework Readiness Frequent design updates RASCI Training (Allrounder) Testing along with BA Team Tools integration with CI Coding guidelines Focus on delivery of Business Value Involving Developers Testing across SCRUM teams Dashboards for Result analysis Test Coverage through unit tests 7

Challenges Faced – Testing Types Automation Testing Performance Testing Security Testing Endpoint URI – Repository outside Codebase Simple Performance Test Environment with Quick Tests Identify and adopt SAST tool at sprint level Junit/Mockito – Repository within Codebase Debugging CI Jobs through tools Plan for DAST at early stage Nightly script runs Result analysis through integrated tools Automation for DAST 8

Conclusion While implementing Dev. Ops concept for one of the payment application, the testing team faced lot of challenges and as a team we were able to come up with appropriate solution. The same has been tried to outline in the paper. The solution are very generic in nature. One may have different challenges and solutions as well considering nature of application, different Dev/Operations/QA team they are working with. We hope this whitepaper would be helpful for the testing team who are yet to start working on testing in Dev. Ops concept. 9

References & Appendix Section Security Testing Challenges Performance Testing Challenges Reference Link https: //www. whitehatsec. com/glossary/content/de vops-testing/? utm_source=onlineadvertising&utm_medium=q 317 -glossary-devopstestinggoogle&utm_source=google&utm_medium=cpc&g clid=EAIa. IQob. Ch. MIkbmfzq 6 G 1 g. IV 0 Es. NCh 1 Wg. LGEAAYBCAAEg. JSdf. D_Bw. E https: //www. blazemeter. com/blog/how-overcomechallenges-when-performance-testing-continuousdelivery-pipelines 10

References & Appendix Terminology QA SAST DAST POC CI IT SME CISQ OMG CDN KPI API Definition Quality Assurance Static Application Security Testing Dynamic Application Security Testing Proof Of Concept Continuous Integration Information Technology Subject Matter Expert Consortium for IT Software Quality Object Management Group Content Delivery Network Key Performance Indicator Application Programming Interface 11

Author Biography Shanthala Bhat is working for Pw. C Limited from last 5 years and currently designated as “Manager”. She holds a degree in E&E engineering from UVCE. Has experience of 12+ years in software testing that includes Testing web applications (server-client network), Web services, Performing Multilingual Testing, Product Testing, Test Management and Test Automation. As a Manager has an expertise in Test Management, development of Test Strategy, test cases, Defect Reports, scope management, risk management and different Test Metrics for V&V activities. Also have knowledge on test automation tools like QTP, VSTS 2012 Ultimate, Selenium, Load. Runner and Telerik. She has written white papers on subjects like Agile Testing, Web Trust & Safety and Framework and Process driven Outsourcing which was selected in one of the STe. P-IN Summit and participated in QAI STC-2016, STe. P-IN Test Autothoon 2017 Contest. She is fun loving person and hobbies include reading magazines, solving cross words, puzzles and listening to music. You can reach her at – Shanthala. ramesh. bhat@pwc. com 12

Co-Author Biography Manjunath Ananthapadmanabha is a Senior Test Lead with Pricewaterhouse Coopers (Pw. C) SDC, Bangalore. He has about 11 years of software testing experience handling projects in specialized testing areas such as Performance, Database, and DW/ETL across numerous industry sectors, including investment banking, healthcare, insurance and telecom. His areas of specialization include Performance testing, continuous integration and test data management. Manjunath has an engineering degree in electronics and communication from Visvesvaraya Technological University, India. Manjunath can be reached at Manjunath. a. padmanabha@pwc. com 13

Thank You!!! 14