Making UAT More Effective Jim Trentadue Software Quality

Making UAT More Effective Jim Trentadue Software Quality Consulting Director Original Software jtrentadue@origsoft. com www. origsoft. com | © Original Software 1

Agenda 1 UAT in IT 2 Defining UAT within the overall test effort 3 Supporting UAT from an IT perspective 4 Enabling UAT Planning 5 Enabling UAT Execution 6 Session Recap www. origsoft. com | © Original Software 2

UAT in IT www. origsoft. com | © Original Software 3

UAT in the IT Industry Role in the various SDLC methodologies Waterfall Agile www. softwaretestingsoftware. com Plan UAT • Define Scope of UAT Identify UAT Scenarios • Review Business Reqs, Use Cases and UI Design • Document UAT Execute UAT Tests • Execute & Document Results Verify UAT defect fixes Report UAT results UAT Signoff • Rerun impacted • Go / No Go Call UAT Scenarios • Log UAT Defects Test Scenarios Test Cases? www. origsoft. com | © Original Software 4

Automation for everyone – even UAT Shift-Left to Reduce Errors, Shift-Right for Better Quality Shift Right! Skillset Spectrum More Technical, More Specialists Pro’s Con’s Cheaper or free software Significant investment to implement Project Team Goal: Do Testing Better! Better-known in Requires market place specialized resources Dev, DBA, QA Specialist No Coding Skills Required Pro’s Con’s Increased utilization of automation Higher entry point cost to buy or build Automation done earlier by everyone Potentially longer to implement BA, Functional QA, UAT www. origsoft. com | © Original Software 5

Defining UAT www. origsoft. com | © Original Software 6

Defining UAT – Testing Assessment Risk Assessment First, understand the risk and strike a proper balance Is there an SQA team? Independent QA team executes system tests? Yes Is UAT the only active testing group? Ensuring we have different roles! www. origsoft. com | © Original Software No Adequate testing time Joint review of artifacts Avoid overlap Coming in too late Production work on hold 7

Defining UAT - Dividing Responsibilities Understanding Breadth vs. Depth testing approaches LOGIN MENU ORDER INVENTORY PROCESSING CONFIRMATION LOGIN • Login as Manager (Full access) • Login as Supervisor (Administrative access) • Login as User (Limited access) MENU • Access order A (Requires full or admin access) (Should be able to access) (Negative test – should not be able to access) ORDER • Process order A • Process order B (Does not require full or admin access) www. origsoft. com | © Original Software REPORTS 8

Supporting UAT www. origsoft. com | © Original Software 9

Responsible parties for UAT Who is responsible for a successful UAT initiative Successful UAT with minimal disruption to the business www. origsoft. com | © Original Software $$$ Business Impact External Customers UAT Lead Environments IT Operations Testing Department Coordination and Training Business Users EVERYTHING! Usability Developers Business Analysts Project Managers Business Schedules Business knowledge and Training Guides 10

Testing Department support for UAT w ie Test Data approach Training outline Execution rt Test Strategy – UAT part Acceptance Test assist po v Re Requirements Stories Design Prototype / Demo Test Plans / Cases Test Summary Retrospectives Defects Planning Re EVERYTHING! Strategy Backlog Status Test Results Defects Product Readiness Scenario sharing Environment preparation Test Data setup Defect retest www. origsoft. com | © Original Software 11

Key questions & concerns Is this just an IT problem? Enough business interest to allot time? Who is accountable for success? How can UAT improve? Technical Enablement www. origsoft. com | © Original Software 12

Enabling UAT Planning www. origsoft. com | © Original Software 13

Enabling UAT Planning: Estimation How should UAT be estimated for planning & execution? Estimation Current Situation Fraction of time taken from QA estimate Plan lacking for other business deadlines Improvement Systematically from UAT regression test runs www. origsoft. com | © Original Software 14

Enabling UAT Planning: Test Assets How much document assistance does SQA provide? Test Assets Current Situation Re-create UAT scripts or take existing SQA tests Commitment from UAT staff on completing testing tasks Improvement Maintain test repository of UAT-created artifacts www. origsoft. com | © Original Software 15

Enabling UAT Planning: Test Mgmt. access Who coordinates what UAT needs / does in TM solution? Test Management Current Situation Unclear what access UAT needs into TM repository System tests / scenarios are given to UAT to modify Improvement UAT specific view & access for relevant information www. origsoft. com | © Original Software 16

Enabling UAT Planning: Test Automation use Who creates / runs the automation and for what purpose? Test Automation Current Situation Test engineer created tests based from system tests Automation is executed by system tester; validation only Improvement Biz process steps capture done by UAT, used by UAT www. origsoft. com | © Original Software 17

Enabling UAT Planning: Training artifacts Does UAT have to re-execute to produce training materials? Training Current Situation UAT focuses on executing test, not for training doc No schedule allocation for training docs production Improvement Enable UAT with markup feature during test execution www. origsoft. com | © Original Software 18

Enabling UAT Execution www. origsoft. com | © Original Software 19

Enabling UAT Execution: Accountability Who has it from IT and Business? Accountability Current Situation Improvement Direct input: IT plans 50%+ resource time for UAT Coordination Biz dedicates colleague for strictly UAT Coordination What is done Who did it When did they Where was it www. origsoft. com | © Original Software 20

Enabling UAT Execution: Testing Status report Who did it? What did they do? Where did they test? Testing Status Current Situation Frequent checkpoint and status updates Lack of time, priority shift; defect ties back to SQA Improvement Clear visibility to UAT results www. origsoft. com | © Original Software 21

Enabling UAT Execution: Continuous Testing How can we turn this into automatic maintainability? Productivity Current Situation Constantly re -engineer test cases for accuracy File location holds test that are outdated; maintenance! Improvement Continuous updates to UAT artifacts; Shift Right! www. origsoft. com | © Original Software 22

Acc ura est te UA ima T tion Re UA usab T te le sts Vie Ded ws icat & W ed ork Le Au verag tom ing atio n Exp o for rt cas Tra es inin g Test Management access TM TA TR www. origsoft. com | © Original Software E Acc bot ounta h IT bil & B ity IZ Tes Dy ting nam Sta ic tus Up d Tes ated t ar UAT tifa cts Solution Implementation / Framework Modification ACC Test Management access tion Test Automation through Test Management access TC UA Exe T cu Test Management access EST Solution Implementation / Framework Modification ing UA Pla T nn Process Review for UAT Planning & Execution Recap of Better UAT procedures REP CT 23

Session Recap www. origsoft. com | © Original Software 24

Presentation Recap Key takeaways for UAT effectiveness Understand where UAT falls within the SDLC and where they are positioned for Test Automation Define where UAT is within the Testing organization and how responsibilities are divided Outline who are the main parties supporting UAT, especially those roles within the Testing department Review the means which UAT Planning can be more efficient, through solutions or framework build in Test Mgmt. system Review the means which UAT Execution can be more effective, through dynamic result reporting and test case maintenance www. origsoft. com | © Original Software 25

Thank you for attending this presentation! Jim Trentadue Software Quality Consulting Director Original Software jtrentadue@origsoft. com www. origsoft. com | © Original Software 26