Test Automation Approach in the Agile World Sammy

Test Automation Approach in the Agile World Sammy Kolluru

Why Test Automation? We want to be more efficient tester We want to build better software We want to deliver faster @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Becoming better & faster How do you find most defects in a product? Test How do you make the tests faster and cheaper? Which tools are useful? @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Automation and the manifesto The increasing necessity of test automation in agile projects results directly from the agile manifesto: (. . . ) "Response to changes Over following a plan" (. . . ) (read: "rapid response to changes"!) @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Regression tests: "The greatest challenge in SCRUM" The effort for regression tests grows significantly with every iteration in Scrum. Strategies for this are risk-based testing and test automation. Factors for an automation decision are: § Frequency of test execution § Automation capability § Maintenance effort @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Which tests to automate? Myths: "Automated tests find more defects. " “All tests can be automated” "Automated tests make manual testers obsolete. " Criteria for which tests should be automated: § Frequent test execution ( regression tests!) § Impossible or difficult to execute manually (load tests) § Fast test execution (sprints are short!) @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Which tests to automate? Typical regression tests: Functional RT: Smoke tests Main (happy) path High risk assessment Nonfunctional RT: Load/Stress tests Performance tests Stability/ Reliability tests @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Agile testing quadrants and test methods business-facing Session-based testing Functional tests Equivalence classes Story tests Boundary values State transistion Examples Status-based test Prototypes Decision tables Simulation Behavior-driven testing Structure-based test Unit tests Interface test Gray box testingtests Component Workflow test Integration tests manual Check the acceptance criteria Exploratory test Scenario tests Defect expectation method Usability tests Heuristic evaluation Acceptance tests Usability lab Pilot run Load profiles Performance Threat scenarios & load tests Failover Endurance test Security tests Stress test "ility" tests Operational testing automated @Sammy. Kolluru | Test Automation Approach technology-facing tools Critiquing the product • The suitable or necessary test methods should be added to the Do. D. Automated & manual Supporting the team • Systematic tests find defects in a reproducible and reliable manner.

What tools are suitable for what? business-facing Session-based testing BDT frameworks (e. g. Functional tests Equivalence classes Cucumber) Story tests (e. g. Limit frameworks values KBT Classification tree Robot Framework, Examples Selenium) Condition-based test Prototypes MBT frameworks Decision tables (e. g. tedeso) Simulation Behavior-driven testing Structure-based test Unit test frameworks Unit tests. Test. NG) Interface (e. g. Junit, test BDT (e. g. Component Gray frameworks box testingtests Rspec) Workflow tests Integration manual Check the acceptance criteria Exploratory test Scenario tests Defect expectation KBT frameworks (e. g. method Usability tests FITnesse, Test. Bench) Heuristic evaluation Acceptance tests Usability lab Pilot run Load profiles Performance & L&P tools (e. g. Jmeter, Threat scenarios Load. Runner) load tests Failover Security test Endurance Security tests frameworks (e. g. Stress test Burp. Suite) "ility" tests Operational testing automated @Sammy. Kolluru | Test Automation Approach technology-facing tools Critiquing the product • The suitable tools and rules for their use should be added to the Do. T. Automated & manual Supporting the team • Depending on test methods different tools are available.

Automation of system tests Data-driven tests Keyworddriven tests Script programming Capture & replay @SAMMYKOLLURU | TEST AUTOMATION APPROACH Frameworkbased automation

Difference in Maturity- Detail Capture/replay Data-driven Keywords Model-based @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Efficient & robust automation. . . Characteristics of immature automations: § High maintenance effort after changes to the SUT § Long duration for creating new automated tests Best practices can prevent this: § Keywords & Data-driven testing § Control over the test environment § Object detection based on unique identifiers @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Nonstop test at the task level User Story READY & divided into tasks Impl. unit test f. other task DONE or Impl. unit test (Test First) Rev. syst. test Coding until unit tests OK Autom. of system tests Bug fix & unit tests Spec. of system tests Rev. of unit tests Autom. of system tests or Execution of system tests Explorat. tests or Developer Tester Coordinated test strategy! System tests for other task @SAMMYKOLLURU | TEST AUTOMATION APPROACH Rev. unit t. f. other task Retest

Automation of all levels Beginning Planung & Planning & control Steuerung test plan, test schedules Analysis & design test specifications, traces for requirements Implementation & execution test specifications, test scripts, test environment, test logs, incident reports Evaluating exit criteria & reporting test (summary) report Test Closure Activities versioning/archive, CIP measures End Figures for a typical system test (“TMAP Next”, chapt. 11. 2) @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Use of tools at all levels Beginning Test Planung & Planning & control Management Steuerung System Analysis & design MBT tools Test Sequencer Implementation & execution Test Roboter Evaluating exit criteria Defect Management & System reporting Test Closure Activities Version Management System End @SAMMYKOLLURU | TEST AUTOMATION APPROACH test plan, test schedules test specifications, traces for requirements test specifications, test scripts, test environment, test logs, incident reports test (summary) report versioning/archive, CIP measures

Positive effects of the use of tools § Automation § § Acceleration and homogenization of work steps Relief and motivation of employees § Functionality § Facilitate activities that cannot be executed manually § Reminder § § Clear, complete documentation Archiving and reproducibility § Communication § § § Uniqueness Completeness Dispersion @SAMMYKOLLURU | TEST AUTOMATION APPROACH

How do you obtain suitable tools? The project-oriented way Impediment Mainten. (by the Scrum Master? ) Proposal for solution with a tool Integration in the team charter Criteria for use Experiment and decision @SAMMYKOLLURU | TEST AUTOMATION APPROACH

General criteria Scalability § Number of users, number and size of artefacts Portability § Support of several platforms and locations Operability § User interface, user guidance and documentation Maintainability § Support for changes in artefacts and tool updates Support § Hotline, support materials, training materials, forums, update frequency Integratability § Compatibility with technical platform, interfaces, adaptability, API @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Criteria for (test) management tools Is it actually necessary to have one or might the agile PM tool be used? Backlog • Traceability between releases, epics, stories and tests • Effort estimate Task board • Status and owner of tests Artefacts • Managing test cases and results • (integration with) version management Burndown • Visualization of the test progress @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Criteria for defect management tools Backlog § Traceability between stories, tests and defects Task board § Status and stakeholder of defects Artefacts § "Rich" defects: attachments, stack traces Analyses § Error progression and distribution Role model § Scrum and/or tester roles § Notification model @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Criteria for test automation tools Points of control and observation • GUI, API, database • Dealing with proprietary interfaces Programming • Capture & Replay • Programming language (proprietary, Java, . NET) • Libraries Process • Distributed tests, scheduler • Unattended testing, integration in Continuous Build, recovery concept Logging • Log files, Screen recording/capturing @SAMMYKOLLURU | TEST AUTOMATION APPROACH

Milestones for a successful introduction of tools Agreement § In the Team § For Scrum of Scrums: Agreement between the teams § For interfaces: with the affected parties (e. g. Product Owner) Uniformity § Control of tool use in the team charter § Identical installation on all computers within the team § For Scrum of Scrums: Re-use of utilization in the teams Use § Regular monitoring of the usefulness in retrospectives § Planning of practice and training measures § Professional problem handling by the Scrum Master (fallback mechanisms, backups, support channels, collecting know-how) @SAMMYKOLLURU | TEST AUTOMATION APPROACH

THANK YOU for attending today’s session @SAMMYKOLLURU | TEST AUTOMATION APPROACH