Transform Manual Testing processes to incorporate Test Automation

Transform Manual Testing processes to incorporate Test Automation Jim Trentadue Software Quality Consulting Director Original Software jtrentadue@origsoft. com www. origsoft. com | © Original Software 1

Agenda 1 Test activities / deliverables by SDLC 2 Automation Models and Frameworks 3 Incorporating Automation into Standard Testing Processes 4 Deep-dive into Testing Artifacts 5 Case Study 6 Presentation Recap www. origsoft. com | © Original Software 2

SDLC Test Activities / Deliverables www. origsoft. com | © Original Software 3

Test Activities within an Agile SDLC Typical Agile process of activities to deliverables Project Initiation Get an understanding of the project Release Planning Participate in sizing stories; Create test plans Each Iteration 1 Participate in sprint planning, estimating tasks Write and execute story tests Pair-test with other testers, developers Business validation (customers) Automate new functional test cases; Run automated regression test cases Run project load tests Demo to the stakeholders … X The End Game (System Test) Release mgmt tests mock deploy on staging; Smoke test on staging Perform load test (if needed) Complete regression test; Participate in releases readiness Business testers perform UAT Release to Prod/Support Participate in release to production Participate in retrospectives www. origsoft. com | © Original Software 4 Agile Testing – Crispin & Gregory

Test activities within a Waterfall SDLC Standard V-Model set of testing activities Project Initiation Test Strategy Test Scenarios Analysis Test Cases Design Test Scripts Develop Test Results Testing Test Summary Deploy www. origsoft. com | © Original Software 5

Test Automation working models and frameworks in the industry www. origsoft. com | © Original Software 6

Test Automation Frameworks Below is the evolution of automation frameworks Record Playback Structured Testing (invokes more conditions) Data Driven Keyword Driven Model / Object Based www. origsoft. com | © Original Software Actions Based Hybrid: combines 2 or more of the previous frameworks 7

Key points with Test Automation & Agile +80%, UI Centric Automation 0 -10% Service or Middle Tier Automation Traditional Automation 0 -10% Agile automation Unit level Automation 0 -10% UI Centric Automation v Smaller units of work built uniquely vs. end-to-end Agile Automation v Constant review of automation backlog v Automation of forms not complete 20 -40% Service or Middle Tier Automation 50 -60% Unit level Automation www. origsoft. com | © Original Software 8 Velocity Partners – The Agile Test Automation pyramid, Mike Cohn

Mapping ATLM against a Waterfall SDLC How an Automated Test Lifecycle Management maps to SDLC C. Small Tool Pilot/Prototype F. Production and Maintenance A. System Life-Cycle Process E. Integration and Test B. Business Analysis and Requirements D. System Design & Development www. origsoft. com | © Original Software 9 Automated Software Testing: Introduction, Management and Performance – Dustin, Rashka, Paul

Fusing Automation into the Process www. origsoft. com | © Original Software 10

Test Automation – Agile fusion Business-Facing Exploratory Testing Scenarios Usability Testing UAT Alpha / Beta Functional Tests Examples Story Tests Prototypes Simulations Unit Tests Component Tests Manual Q 2 Q 3 Q 1 Q 4 Critique Product Supporting the Team Automated & Manual Performance & Load Testing Security Testing “ility” Testing Automated Tools Technology-Facing www. origsoft. com | © Original Software 11 Agile Testing – Crispin & Gregory

Test Automation – Agile (ish) fusion No one admits to doing pure Waterfall anymore Test Automation activities: Define TA scope in Release Test Plan Project Initiation Test Strategy Test Scenarios Automated & Manual Scenarios Analysis Prepare error-handling (new and regression) Design Test Cases Record carefully Develop Replay Testing Test Scripts Test Results CM Test Summary Deploy www. origsoft. com | © Original Software 12

Deep-dive into Testing Artifacts www. origsoft. com | © Original Software 13

Test Strategy / Test Plan Avoid Plan To obtain buy-in for automation Omitting automation Not listing automation for accomplishing the testing on schedule How the tool selection was done and how testers will use this Omitting variables: Resources, Data, Objects, Licenses, Environments To include automation in every iteration / build www. origsoft. com | © Original Software 14

Test Stories / Test Scenarios Avoid Topics not automatable Plan To parse candidates Planning at a wrong level for automation To define the ‘what’ along with ‘how’ Ambiguous areas: Usability, Reliability, Maintainability, etc. Peer reviews to ensure accuracy in automation www. origsoft. com | © Original Software 15

Test Scripts / Test Cases Avoid Plan Better Tests; Shift. Right Automation as a finality Planning automation until late in schedule Frequent automated test runs from the onset Delaying automation creation & execution at start of schedule/sprint End-to-end regression test runs for changes www. origsoft. com | © Original Software 16

Retrospectives / Test Summary Avoid Not doing tweaks Plan To improve automation The notion that TA is like a manual test case TA metric reviews per iteration / sprint Not scheduling a maintenance window To automate completed manual tests www. origsoft. com | © Original Software 17

Case Study www. origsoft. com | © Original Software 18

Background & Solution Titled: Automation Through the Back Door (Manual Test Support) Background: To improve the rate of automation, changes to the TA framework were required to support manual testing Technical Solution: Develop framework called command-driven testing DRIVER-File DATA-File GOTO, Customer Window INPUT, First. Name , <First. Name> INPUT, Name <Last. Name> INPUT, Address <Address> INPUT, Bank Account <Account. No> SELECT, Button, OK CLOSE <First. Name>, “John” <Last. Name>, “Doe” <Address>, “ 1 Main Dr. ” What is Command-Driven Testing? • Uses simple command keywords (SELECT, BUTTON) • Interpreter scripts are the same for all product Script-Runner Test Execution • Using advantages from Data-Driven testing, navigation was in a DRIVER-File; data in a DATA-File Script-Runner Test Tool Loop thru statements GOTO Interpret & execute INPUT Interpret & execute SELECT • Two files built together form a command script • Sequentially, the script-runner reads the commands in the DRIVER-File. DATA-Codes are substituted with data from the DATA-File With Command-Driven testing: • Testers don’t need to learn tool scripting • Separation in navigation & data make command scripts flexible & reusable • DRIVER-Files can be ported to different apps & can migrate to another tool • The test tool is needed only to prepare the templates and to run the tests www. origsoft. com | © Original Software 19 Experiences of Test Automation – Graham & Fewster

Process for Implementation STEP 1 Prerequisites: 1) Can’t start without knowledge of application under test and test cases to be automated Prereqs 1. Test Cases 2) Need working engine to interpret commands for all test cases 2. 3) All GUI elements used in execution must be registered by the capture / replay tool for object name normalization 3. Automator →Capture→Tool Script Automated Test Suite Command Library Registration of GUI Controls STEP 2 ↓ Tool Script ↓ Machine ↓ Command Script Test Engine ↓ STEP 3 Command Script →Tester→Suite Test Results Step 1 Step 2 Step 3 Record the TC with the capture functionality of automation solution Translate generated script to command-script Develop TC’s in template and build test suites www. origsoft. com | © Original Software 20 Experiences of Test Automation – Graham & Fewster

Case Study - Key Points Command –Driven framework was adopted for manual testing as well Helped limit testers working on same template concurrently Defect reporting was much easier with rework to recreate the issue Constant reviews to assess ongoing needs for automated and manual testing Implementation supported partial automated test execution; for test prep tasks Manual tests now focused on customer-specific conditions instead www. origsoft. com | © Original Software 21

Presentation Recap www. origsoft. com | © Original Software 22

Recap of the Presentation Outline your process by phase and deliverable; insert automation often Research industry frameworks and apply best practices Fuse automation topics into each testing phase with corresponding deliverable Delve into your testing process to plan and avoid topics around automation Present a real-life case study to management to build your case www. origsoft. com | © Original Software 23

Thank you for attending this presentation! Jim Trentadue Software Quality Consulting Director Original Software jtrentadue@origsoft. com www. origsoft. com | © Original Software 24