Paratus Systems Aundh Software Testing Paratus Systems Aundh

Paratus Systems, Aundh Software Testing

Paratus Systems, Aundh Agenda • Principles of Software Testing • Testing Strategies • Advanced Techniques and Tools • Assignments 2

Paratus Systems, Aundh Principles of Software Testing Definition • Testing • It is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements – [IEEE 83 a] • Process • Sequence of steps performed for a given purpose [IEEE] • Software Process • A set of activities, methods, practices and transformations that people use to develop and maintain software and associated products [SEI-CMM] 3

Paratus Systems, Aundh Principles of Software Testing Objectives • Primary: • Execute a program with a intent of finding errors to • Determine whether system meets specification • Determine whether system meets user’s needs • Secondary: • Continuously improve the testing process 4

Paratus Systems, Aundh Principles of Software Testing Misunderstanding about Testing • Testing is debugging • Testing is not the job of a programmer • If programmers are more careful testing would be unnecessary • Testing never ends • Testing activities start only after the coding is complete • Testing is not a creative task 5

Paratus Systems, Aundh Myths and Facts about testing Testing is a single phase in SDLC. Testing is easy. Development is worth more than testing. Complete testing is POSSIBLE. Anyone can be a Tester. It’s a continuous process. Test cases are designed manually. Testing works throughout SDLC. All inputs & lines cannot be tested. Training & skills are required. 6

Paratus Systems, Aundh Principles of Software Testing Theory vs Reality • Theory: • Need Sufficient Time • Base-lined and Frozen Requirements • Automation Testing • Have trained testers: test process, business flow, … • Reality: • Time Pressures to keep delivery date • Continuously updated requirements • Don’t know when to stop: Update Requirements continuously (CRs) -> Testers update TC and re-tests • Manual Testing • Lack of dedicated software test environment: share, steal • Inexperienced testers without appropriate training 7

Paratus Systems, Aundh Principles of Software Testing • The goal of testing is to find defects before customers find them out • Exhaustive testing is not possible; program testing can only show the presence of defects, never their absence • Testing applies all through the software life cycle and is not an end-of-cycle activity • Understand the reason behind the test • Test the tests first • Tests develop immunity and have to be revised constantly • Defects occur in convoys or clusters and testing should focus on these convoys • Testing encompasses defect prevention • Testing is a fine balance of defect prevention and defect detection • Intelligent and well-planned automation is key to realizing the benefits of testing • Testing requires talented, committed people who believe in themselves and work in teams 8

Paratus Systems, Aundh Principles of Software Testing Verification and Validation Criteria Verification Validation Definition The process of evaluating workproducts (not the actual final product) of a development phase to determine whether they meet the specified requirements for that phase. The process of evaluating software during or at the end of the development process to determine whether it satisfies specified business requirements. Objective To ensure that the product is being built according to the requirements and design specifications. In other words, to ensure that work products meet their specified requirements. To ensure that the product actually meets the user’s needs, and that the specifications were correct in the first place. In other words, to demonstrate that the product fulfills its intended use when placed in its intended environment. Question Are we building the product right? Are we building the right product? Evaluation Items Plans, Requirement Specs, Design Specs, Code, Test Cases The actual product/software. Activities • Reviews • Walkthroughs • Inspections • Testing 9

1. Verification is a static practice of verifying documents, design, code and program. 2. It does not involve executing the code. 3. It is human based checking of documents and files. 4. Verification uses methods like inspections, reviews, walkthroughs, and Desk-checking etc. 1. Validation is a dynamic mechanism of validating and testing the actual product. 2. It always involves executing the code. 3. It is computer based execution of program. 7. Target is requirements specification, application and software architecture, high level, complete design, and database design etc. 8. Verification is done by QA team to ensure that the software is as per the specifications in the SRS document. 9. It generally comes first-done before validation. 7. Target is actual product-a unit, a module, a bent of integrated modules, and effective final product. 8. Validation is carried out with the involvement of testing team. 4. Validation uses methods like black box (functional) testing, gray box testing, and white box (structural) testing etc. 5. Verification is to check whether the software 5. Validation is to check whether software meets conforms to specifications. the customer expectations and requirements. 6. It can catch errors that validation cannot catch. 6. It can catch errors that verification cannot It is low level exercise. catch. It is High Level Exercise. 9. It generally follows after verification. 10

Verification Methods Audit-Quality check Inspection-Author, moderator, reader, tester Walkthrough Review 11

Paratus Systems, Aundh Principles of Software Testing V-test Model 12

Paratus Systems, Aundh Principles of Software Testing Defect Management 13

Paratus Systems, Aundh Testing Strategies • It includes identifying • What type of testing would you use for testing the functionality? • What are the configurations or scenarios for testing the features? • What integration testing would you do to ensure these features work together? • What localization validations would be needed? • What “non-functional” tests would you need to do? 14

Paratus Systems, Aundh Testing Strategies Unit Testing • It is a level of the software testing process where individual units/components of a software/system are tested • The purpose is to validate that each unit of the software performs as designed • It is a method by which individual units of source code are tested to determine if they are fit for use • It is concerned with functional correctness and completeness of individual program units • It is typically written and run by software developers to ensure that code meets its design and behaves as intended • Its goal is to isolate each part of the program and show that the individual parts 15 are correct

Paratus Systems, Aundh Testing Strategies Integration and System Testing • Testing the interaction between the modules and interaction with other systems externally is called integration testing • Integration testing starts when two of the product components are available and ends when all component interfaces have been tested • Integration testing is both a type of testing and a phase of testing • The testing conducted on the complete integrated products and solutions to evaluate system compliance with specified requirements on functional and nonfunctional aspects is called system testing • It is the first level of software testing where the application is tested as a whole • Non-Functional aspects include Performance/Load testing, Scalability testing, Reliability testing, Stress testing, Interoperability testing, Localization testing 16

Paratus Systems, Aundh Testing Strategies Acceptance Testing • Acceptance testing is done by the customer or by the representative of the customer to check whether the product is ready for use in the real-life environment • Following test cases can be included for acceptance testing • End-to-end functionality verification • Domain tests • User scenario tests • Basic sanity tests • New functionality • A few non-functional tests • Tests pertaining to legal obligations and service level agreements • Acceptance test data 17

Paratus Systems, Aundh Testing Strategies Alpha and Beta Testing • Alpha testing performed at developers site but not by developing team • Testing a software product which is not the final version. • This software does not have to necessarily contain the full functionality required for an application, however core functionality to accept input an generate output is required. • Here virtual environment will be created to simulate real time environment. • Beta or field testing is done by customers at their locations • Beta Testing is last stage of testing where a product is sent outside the company or offer the product for free trial download 18

Paratus Systems, Aundh Testing Strategies Performance Testing • The testing performed to evaluate the response time, throughput and utilization of the system, to execute its required functions in comparison with different versions of the same product(s) or different competitive product(s) is called performance testing • It is done to ensure that a product • Processes the required number of transactions in any given interval (throughput) • Is available and running under different load conditions (availability) • Responds fast enough for different load conditions (response time) • Delivers worthwhile return on investment for the resources – hardware and software – and deciding what kind of resources are needed for the product for different load conditions (capacity planning) • Is comparable to and better than that of the competitors for different parameters (competitive analysis and bench marking) 19

Paratus Systems, Aundh Testing Strategies Security Testing • Security testing is a process to determine that an information system protects data and maintains functionality • To check whethere is any information leakage • To test the application whether it has unauthorized access and having the encoded security code • To find out all the potential loopholes and weakness of the system that can cause loss of important information • To identify the vulnerabilities and subsequently repair them • It helps in improving the current system and ensuring that the system will work for longer time 20

Paratus Systems, Aundh Testing Strategies White Box and Black Box Testing • White box testing is a way of testing the external functionality of the code by examining and testing the program code that realizes the external functionality • It is also known as clear box, or glass box or open box testing • It takes into account the program code, code structure and internal design flow • Black box testing is done without the knowledge of the internals of the system under test • It is done based on requirements • It addresses the stated requirements as well as implied requirements • It encompasses the end user perspectives • It handles valid and invalid inputs 21

Paratus Systems, Aundh Testing Strategies Basis path testing • Basis path testing, a structured testing or white box testing technique used for designing test cases intended to examine all possible paths of execution at least once. • Creating and executing tests for all possible paths results in 100% statement coverage and 100% branch coverage. • Example: Function fn_delete_element (int value, int array_size, int array[]) { 1 int i; location = array_size + 1; 2 for i = 1 to array_size 3 if ( array[i] == value ) 4 location = i; end if; end for; 5 for i = location to array_size 6 array[i] = array[i+1]; end for; 7 array_size --; } 22

Paratus Systems, Aundh Testing Strategies Basis path testing • Steps to Calculate the independent paths • Step 1 : Draw the Flow Graph of the Function/Program under consideration as shown below: • Step 2 : Determine the independent paths. • Path 1: 1 - 2 - 5 - 7 • Path 2: 1 - 2 - 5 - 6 - 5 - 7 • Path 3: 1 - 2 - 3 - 2 - 5 - 6 - 5 - 7 • Path 4: 1 - 2 - 3 - 4 - 2 - 5 - 6 - 5 - 7 23

Paratus Systems, Aundh Testing Strategies Equivalence testing • Equivalence partitioning is a software testing technique that involves identifying a small set of representative input values that produce as many different output conditions as possible • It is useful to minimize the number of test cases when the input data can be divided into distinct sets, where the behavior or outcome of the product within each member of the set is the same • Example: Identify equivalence classes for this requirement “If a pupil has total score >=75, he will pass the exam, otherwise will fail (total score is an integer)” • Search Function Requirement: User fills in search field then clicks search. The system will find all search strings in the document and highlight them. If none is found, message on form to indicate no results found (search like Notepad) • Savings Account: 3% rate of interest is given if the balance in the account is in the range of $0 to $99. 99, 5% rate of interest is given if the balance in the account is in the range of $100 to $999. 99, and 7% rate of interest is given if the balance in the account is $1000 and above, identify three valid equivalence partitions and one invalid partition. 24

Paratus Systems, Aundh Testing Strategies Graph Based testing • Graph based testing methods are applicable to generate test cases for state machines such as language translators, workflows, transaction flows and data flows 25

Paratus Systems, Aundh Testing Strategies Test Metrics • Metrics derive information from raw data with a view to help in decision making. • Metrics can be classified as product metrics and process metrics • Product metrics can be further classified as • Project metrics • Effort variance, Schedule variance, Effort distribution • Progress metrics • Testing Defect Metrics • Defect find rate, Defect fix rate, Outstanding defects rate, Priority outstanding rate, Defects trend, Defect classification trend, Weighted defects trend, Defect cause distribution • Development Defect Metrics • Component-wise defect distribution, Defect density and defect removal rate, Age analysis of outstanding defect, Introduced and reopened defects rate • Productivity metrics • Defects per 100 hrs of testing, Test cases executed per 100 hrs of testing, Test cases developed per 100 hrs, Defects per 100 test cases, Defects per 100 failed test cases, Test phase effectiveness, Closed defect distribution 26

Paratus Systems, Aundh Testing Strategies Test Reporting • Test incident report It is a communication that happens through the testing cycle as and when defects are encountered • Test cycle report It gives • A summary of activities carried out during that cycle • Defects that were uncovered during that cycle, based on their severity and impact • Progress from the previous cycle to the current cycle in terms of defects fixed • Outstanding defects that are yet to be fixed in this cycle • Any variations observed in effort or schedule • Test summary report • Phase-wise test summary, which is produced at the end of every phase • Final test summary reports, which has all the details of all testing done by all phases and teams, also called as “release test report” 27

Paratus Systems, Aundh Advanced Techniques and Tools GUI testing • Graphical User Interface [GUI] Testing – methods used to identify and conduct GUI tests, including the use of automation tools • Elements of GUI Testing – • A process • A GUI Test Plan • A set of supporting tools • Running the Test Cases – • Mouse position capture • Event capture 28

Paratus Systems, Aundh Advanced Techniques and Tools Functional testing • It involves product features and functionality • It tests product behavior • Result conclusions are simple steps written to check expected results • Results vary due to product implementation • Testing focus is on defect detection • Product and domain knowledge are required for testing • Failures are normally due to code • Testing phases include unit/component, integration and system testing • One-time setup is required for a set of test cases • Test cases are executed many times 29

Paratus Systems, Aundh Advanced Techniques and Tools Automated Testing Tools - features • Reliable : Tests perform precisely the same operations each time they are run, there by eliminating human error. • Repeatable: We can test how the application reacts after repeated execution of the same operation • Comprehensive : We can build a suite of tests that covers every feature in our application • Reusable : We can reuse tests on different versions of an application, even if the user interface changes. • Better Quality Software: Because you can run more tests in less time with fewer resources • Fast: Automated Tools run tests significantly faster than human users. • Cost Reduction: As the number of resources for regression test are reduced 30

Paratus Systems, Aundh Advanced Techniques and Tools Automated Testing Tools - selection Step 1: Define Tool requirements based on Project’s testing strategy and: Business requirements Tool audience Budget constraints Step 2: List possible Tool Candidates in the required category & collect Tool Data Step 3: Compare each tool with its basic features against the Tool requirements defined in Step 1 and the cost Step 4: Evaluate/Rank the Tools Step 5: Select or recommend Tool with highest ranking / best pricing Step 6: Arrange a Demo of Tool in Test Environment Step 7: Finalize 31

Paratus Systems, Aundh Advanced Techniques and Tools Mobile Testing • Mobile Application Testing is a process by which a developed application of a mobile device is tested for its functionality, consistency and usability • Challenges involved include: • Wide varieties of mobile devices • Different mobile operating system • Different mobile network operators • Input method • Hardware compatibility • Type of testing include: • Functional testing • Laboratory testing • Performance and Load testing • Security testing • Localization testing • Usability testing 32

Paratus Systems, Aundh Advanced Techniques and Tools Testing Tools - Selenium • Selenium is a Functional Automation Tool for Web Applications • It is an open source tool (no cost involved with it) • It supports the languages like HTML, Java, PHP, Perl, Python, Ruby and C# • It supports the browsers like IE, Mozilla Firefox, Safari, Google Chrome and Opera • It supports the operating systems like Windows, Linux and Mac • It is very flexible when compared to QTP and other functional tools, because it supports multiple languages • It is mainly built on 3 components: • Selenium IDE • Selenium RC • Selenium Grid 33

Paratus Systems, Aundh Advanced Techniques and Tools Testing Tools - JUnit • JUnit has become the standard tool for Test-Driven Development in Java • JUnit test framework is a package of classes that lets you write tests for each method, then easily run those tests • Test. Runner runs tests and reports Test. Results • You test your class by extending abstract class Test. Case • To write test cases, you need to know and understand the Assert class • To start using Junit, create a subclass of Test. Case, to which you add test methods • Name of the class is important – should be of the form Test. My. Class or My. Class. Test • This naming convention lets Test. Runner automatically find your test classes • Each assert method has parameters like: message, expected-value, actual-value 34

Paratus Systems, Aundh Advanced Techniques and Tools Testing Tools – monkey talk • Previously named “Fone. Monkey” • Tool for automated testing of i. OS, Android, HTML 5 and Adobe Flex applications • It is an integrated environment for recording, customizing, running and managing test suites • Free and Open Source • Powerful and robust record and playback functions for mobile applications • Powerful IDE • Fully cross platform • No jailbreaking needed 35