Choosing the right Test Automation Framework Agenda About

Choosing the right Test Automation Framework

Agenda About Test Automation Framework How the test automation framework should be - Benefits Factors to be consider while choosing test automation framework Understand the types of test automation frameworks Examples : Real time framework designs Conclusion

Test Automation Frameworks • Set of assumptions, concepts and practices that constitute a work platform or support • Process of structuring and organizing scripts to increase the level of reusability and readability • Independent of application / environment under test • Reusable set of libraries or classes for a software system or subsystem.

Benefits of Frameworks Ease of use – easy to learn and easy to use Time – Faster than capture /replay and scripting approach Maintainability – Significantly reduces the test maintenance effort Reusability – Due to modularity of test cases and library functions Manageability – Effective test design , execution and traceability Accessibility – To design , develop and modify test while executing Availability – Scheduled execution can run unattended on a 24/7 basis

Factors to be considered while choosing test Automation Frameworks • Who are the end users • How can it be made easily understandable Clear vision • Flexibility to enhance & scope • Strong perspective of Software Design and Architecture Building • Software architect accompanied by developers and Test Automation experts framewor k team • A framework should be one that is easy to adopt and fairly intuitive for new users Usage and • It should bring basic standardization and uniformity to build robust and scalable test automation. Adoption • Framework is living artifact which we should be able to enhance to cater the ongoing needs Ongoing • Proactive teams that understand the critical success factors of the framework need to constantly keep pace with the change Maintena requirements nce • Framework development and maintenance costs need to be budgeted, justified and monitored thoroughly and frequently Cost Considerat ions

Types of test automation frameworks Record and playback Linear scripting Modular scripting Test data driven Hybrid driven testing Keyword driven testing Acceptance Test Driven Development (ATDD) Behavioral Driven Development(BDD)

Record and Playback • Capture interaction with system and replay it. • Popular approach among commercial tools • Examples:

Linear Scripting • Non-Structured Scripts interact directly with the system under test • Can use any programming language • Also produced by capture and replay tools • Examples:

Modular Scripting • Driver scripts “drive” test execution • Interaction with the SUT done by functions in a test library

Modular Scripting – Examples

Data Driven Testing • Test data taken out of test scripts § Customarily presented in tabular format • One driver script can execute multiple similar tests • New driver script still needed for different kinds of tests

Keyword Driven Testing • Not only test data but also directives (keywords) telling how to use the data taken out of the test scripts • Keywords and the test data associated with them drive test execution

Hybrid Test Automation Framework Error Logs Test data(Exc el) Env Details Test reports Email Report Layer Data Layer Re-usable Functions Test Suite AUT Obj-1 Obj Repo Obje-2 Tests Data base layer

Acceptance Test Driven Development Framework (ATDD) • Collaboration between Business Analysts , QA Engineers , Developers to ensure that the requirements are well defined • A set of concrete acceptance tests before development begins. • This is the best of way to ensure that we have the all same shared understanding of what it is we’re actually building Review Acceptance Criteria (PDM, BA, SA & QA) Coding & Unit Testing (Dev) Develop FF (Dev &QA) Sign-off Story(PD M, SA, De v &QA) Sign-off FF (PDM, SA, Dev &QA) Review FF (PDM, SA, DEV, QA &QA) Step Definition or Functional Tests(QA)

Acceptance Test Driven Development Framework Contd. ,

Behavior Driven Development Framework (BDD) • Collaboration between Business Analysts , QA Engineers , Developers and business owners/Stakeholders • Extends TDD by utilizing natural language that non-technical stakeholders can understand • BDD Frameworks such as Cucumber or Behave are an enabler , acting a “bridge” business and technical language

BDD Example : Feature (Behavioral Description)

Real Time : BDD Hadoop Data SQL Data verification and validation Execution Env Project Config Details Jenkins Build Data Jenkins Reports Csv reports Reports Step Definitions Core Config Read test data from yml files Execution Logs and writing mismatched records to csv Data Matchers raise Data Reader Data Writer Verifiers • hadoop_info • Oracle_db_info • db_queries • Cucumber properties

Real Time : Hybrid Framework

Real Time : Keyword Driven Framework svn checkout: http: //csvtodbcompare. googlecode. com/svn/trunk/ Google : csvtodbcompare

Conclusion • The process of automation framework design and development requires detailed planning and effort. To achieve the desired benefits, the framework must be accurately designed and developed. Such a framework can used across all the projects in an organization and provides substantial ROI • When choosing an automation framework , it is crucial to ensure that it can easily accommodate various automation testing technologies and changes in the system under test

Thank You!