Digital Performance Testing Preeti Kambli Manager Venkata Goday

Digital Performance Testing Preeti Kambli, Manager Venkata Goday, Director Capgemini 1

Abstract q 89% of companies will primarily compete on the basis of customer and Omni channel experience q CIOs are shifting their investment pattern in response to digital business, with average already spending 18% of their budget for digitalization and set to increase to 28% by end of 2018 • Key focus areas for IT organizations - Digitization and customer experience. • Digital ecosystem involves enterprises, partners, customers and other stakeholders who experience the Information systems through multiple mediums and access points. • Comparisons are quickly drawn based on their interactions, hence its outmost important for IT organizations to provide highest user experience levels. The single source for validating the systems responsiveness is by thoroughly performance testing. • This paper talks about the customer user expectations and our solution for digital performance testing 2

Introduction Digital disruption in the today’s age is accelerating Application performance is all about end user experience in the digital era, irrespective of the technology or complexity. With the evolution of Mobiles, Tablets and handheld devices it has become imperative to stay ahead in terms of understanding user experience even before users actually experience it. IT Transformation silo Business silo Need to break the silo and have a more connected customer demand; end • behavioural analysis user expectation • social sentiment analysis • heat maps • voice of customer data • • system health load time MTTR, TTFB, SDLC Latency and release cycles 3

Challenges in digital performance testing Traditional performance testing practice is more server focused and doesn’t consider end user responsiveness A complex mix of devices, operating systems, network, ever emerging technologies and most importantly – customer expectation makes it difficult to “fix” issues as soon as the customer experience them Often end to end performance tests are bypassed owing to shorter release cycles & quicker time to market – ultimately resulting in unknown performance issues & bad user experience in Production Infrastructure availability for load simulation remains challenge given the large Omni channel spread Lack of standard approaches, frameworks and benchmarks for validation purposes 4

Factors affecting performance Virtual users & monitoring 5

Client side application performance Native mobile application § check on how much of the application and its data resides on the local device and communicate readily with a server application § Device’s own hardware and software configuration come into play Mobile browser-based application performance is usually heavily dependent on network and server application performance Server application needs to be able to recognize the device/browser combination in order to render properly 6

Server side performance Examining the server performance is similar to measuring website or web app performance Need to decompose the analysis into the components of the server that are providing the services including the database, application server, and associated hardware Architecture has many variables that can result in numerous interaction between its components which could significantly impact performance 7

Network performance Network protocols impact throughput and delays Test on different networks and in different countries Carriers place overhead on § data transmission § network latency Components to be measured § Traffic b/w Client (desktop, device) & Web/App § App Server & DB Attributes § Bites Transferred § Connection Failures § Bandwidth Utilization § Transmission Errors 8

Tool & Techniques No single tool can solve all parts of the problem statement for digital performance as most tools are specialized towards a specific platform and testing aspect Many commercial and open source performance testing tools available for desktop browser based apps Limited options for native mobile apps Omni channel apps end to end performance testing involves multiple devices, OS versions, app versions, and different servers for native and web applications. Functional mobile testing solutions simulate user actions and executing running on different devices. Test tool automation framework help in validating the functional aspect of the application on device. How to integrate functional and performance tools to report on a real world end user digital experience? 9

APM tools for Omni channel performance monitoring Real user monitoring (RUM) capabilities for Mobile and Browser capabilities providing business insights, user behavior analytics, root case analysis, error rates User Journey Page breakdown metrics Mobile/PC/Browser Number of Sessions User response times Network providers Conversion/Bounce rates Problem detection Alerting Tools – Dynatrace, New Relic, App. Dynamics Web page diagnostics tools Identify and diagnose web page and URL performance issues; elements that cause sites to hang and slow down. These issues give users the perceived notion of SPA and website slowness and unresponsiveness. Tools - Google Developers Chrome Dev. Tools, Web Page Analyzer, HPE Network Virtualization (NV) Analytics 10

Performance tools for server side metrics Measure key KPIs like response times, throughput, transaction/second, hits/second and provide a first glance of the server behaviour under load. Tools - HPE Load. Runner, SOASTA, Neo. Load, Jmeter, Blaze. Meter Device side performance solution/tools Improve the quality of your i. OS, Android, and web applications by testing against browsers and real mobile devices hosted at the data center or on the cloud. Optimize performance by simulating real world network conditions and monitoring device vitals consumption (CPU, memory, and battery) Tools - HPE Mobile Center, Perfecto Mobile, Experitest Saa. S digital assurance labs, Mobile Labs device. Connect Network performance tools Capture and emulate real-world network conditions to detect and remediate issues before app deployment. Analyze results and gain insight into the root cause of network performance bottlenecks Tools - HPE Network Virtualization (HPE NV), Op. Manager 11

Case Study. Digital Performance Testing for a Leading Media & Entertainment client Business drivers Measure response time experienced by its end users through the recently launched digital delivery platform for marketing of entertainment and news to global audience Define simulation strategy to include device type/configuration, operating system, user base across geography, bandwidth/network and latency Track real user and synthetic test interactions to benchmark performance from different geographies Test environment included setup of dedicated Mobile lab with multiple device types and configurations to simulate browser and native app real time simulations Business Transaction Management 12

Device side Perf Monitoring Mobile Labs device. Connect Omni channel Emulated Devices / Operating Sys Load Generators Automated performance + functional Script Server side Performance HP Load. Runner Hybrid Mobile Requests Generating Concurrent User Load Unified solution for server side, device side, APM and functional test Network Emulation – Varied Bandwidth Network performance HPE NV Monitoring Engine 3 G 4 G Wifi tion lica p p A s res tric ptu ce Me a C l n Too forma r e P Application, Middle Tier and Database Profiling Tools Test Console M/C (Controller) Application Scenario Recording & Scripting Mobile Apps Performance Test Results + Devise stats Application Profiling Metrics & Reports Analysis Device side Perf Monitoring Mobile Labs device. Connect Data Center - Application Under Test APM for Monitoring / Diagnostics Appdynamics Performance Test Reports &Tuning Recommendations Performance tuning was carried out for the top time consuming interactions Identified “Out. Of. Memory” exception due to continuous usage of large object heap memory Identified repetitive calls for same elements/images on same page & were optimized 13

Summary Need to understand the customer demands and emotions Focus on simulating a real world scenario for performance Assess the various simulation tools and choose the right tool Cohesive digital ecosystem combining the perfect blend of server side, device side, network, APM with a pinch of functional testing touchpoint is equally important. Adoption of functional and device side scenarios will play a critical role in making business decision and having an accurate end user experience measured out of your digital performance testing. 14

References & Appendix • • • https: //www. slideshare. net/Ry an. Bateman 4/what-is-digitalperformance-management http: //www. methodsandtools. com/archive/mobiletest 1. html https: //experitest. com/mobilecloud-testing/seetestcloudonline/? am_force_theme_lay out=desktop https: //engineering. linkedin. co m/blog/2017/02/measuringand-optimizing-performanceof-single-page-applications https: //en. wikipedia. org/wiki/S ingle-page_application http: //www. techrepublic. com/ blog/web-designer/freediagnostic-tools-for-websiteresponse-and-performanceissues/ Abbreviation Full form IT Information Technology CIO Chief Information Officer API Application Programming Interface UI User Interface SDLC Software development Life Cycle MTTR Mean Time To Repair TTFB Time To First Byte SPA Single Page Application HTML Hypertext Markup Language APM Application Performance Management HPE Hewlett Packard Enterprise (now Microfocus) UFT Unified Functional Testing CPU Central Processing Unit Co. E Center of Excellence 15

Author Biography Preeti Kambli Performance Testing & Engineering Co. E Lead for Capgemini Mumbai 15 year experience in business development support; architecting solutions for clients, competency building for various performance tools and enabling delivery projects in crisis situation Ideated and managed the development of new accelerators for Performance testing and drive the asset industrialization campaign. Venkata Goday Performance Testing & Engineering Co. E Head for Capgemini global Leading this practice from the last decade well known thought leader in performance engineering Rich experience in setting up Performance Co. E’s for customers and has partnered with several clients for strategic consulting and transformation initiatives Architected and developed many assets in Non-Functional testing space. 16

Thank You!!! 17