Report on ISTQB testing course Gianni Pucciani ITGT

Report on ISTQB testing course Gianni Pucciani, IT-GT CERN, White Area Lecture, 1 October 2010

Outline �General information about the course �Contents and questions �Feedback from participants 2 Gianni Pucciani, ISTQB course report 1 October 2010

ISTQB certification scheme � International Software Testing Qualifications Board � http: //istqb. dedicated. adaptavist. com/display/ISTQB/Ho me � More than 130 K certified “testers” � Syllabus available on-line http: //www. softwaretester. ch/english/index_en. ht ml 3 Gianni Pucciani, ISTQB course report 1 October 2010

Course overview �First software testing course at CERN (right? ) � 4 days with final exam, based on new 2010 syllabus. �http: //www. synspace. com/en/ �Material: slides, book, exercises, tools list � 12 participants �Not much space for technical details outside the syllabus, though instructor was available and expert enough to answer specific questions. �Final exam: � 75 minutes, 40 multiple choice questions, at least 4 Gianni Pucciani, ISTQB course report 65% to pass. 1 October 2010

Course contents �Fundamental of testing �Testing throughout the SDLC �Static testing �Test design techniques �Test management �Tool support for testing 5 Gianni Pucciani, ISTQB course report 1 October 2010

Fundamental of testing � What is testing � “The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. ” (ISTQB glossary V 1. 2) �Error, defect, failure � General testing principal �Early testing, pesticide paradox, defect clustering, testing can show the presence of defects but not their absence � Fundamental test process �Process �Exit criteria: coverage, error discovery rate, time & resources � The psychology of testing �Independence of testers, bug report � Question 6 Gianni Pucciani, ISTQB course report 1 October 2010

Testing throughout the SLC �Review of main Software Development Models �Test Levels �Component Integration System Acceptance �Integration plan should drive the development and testing plans �System testing, ISO 9126 for quality characteristics �Maintenance testing, testing changes of a running system. �Test types or targets �Question 7 Gianni Pucciani, ISTQB course report 1 October 2010

Static testing � Static testing in the test process �It applies to all the phases of the SDLC, complementary to dynamic testing � Structured group examinations �Phases (planning, kick-off, individual preparation, review meeting, rework, follow-up) and roles (manager, moderator, author, reviewers, scribe) �Review types �Cost of reviews � Static analysis �Cyclomatic complexity � Question 8 Gianni Pucciani, ISTQB course report 1 October 2010

Test design techniques � Test case design � test case specification: identifier, input, output… � Black-box � Equivalence class � Boundary value (testing boundaries of equivalence classes) � Decision table for rule based requirements � State transition testing for FSM � Use case testing � White-box � Statement coverage, decision coverage, condition coverage, multiple condition coverage, condition determination coverage � Exploratory � Based on practical experience and knowledge of the test object. � http: //www. satisfice. com/blog/ � Choosing design techniques � Question 9 Gianni Pucciani, ISTQB course report 1 October 2010

Test management � Test (team) organization �Different way of organizing developers and testers � Test planning and estimation �Risk based approach (likelihood and impact) test prioritization �Test plan, IEEE 829 �Early failure detection � Test progress monitoring and control �metrics � Configuration management � Incident management �basics and importance for process improvement � Norms and standards � Question 10 Gianni Pucciani, ISTQB course report 1 October 2010

Tool support for testing �Types of test tools �Use of test tools, benefits and risks �Time, cost, effort for introduction, compatibility with organization processes, attitude of people… �Interoperability �Selection and introduction of test tools �Introduce tools only in a mature environment �Question 11 Gianni Pucciani, ISTQB course report 1 October 2010

Feedback from participants �Instructor and material very good �Program, very well received with few (1? ) exceptions. �Pros: common basis, speak the same language. �Cons: no practical details, known issue though. �Follow-up �Workshop style, sharing CERN experience. �Advanced level 12 Gianni Pucciani, ISTQB course report 1 October 2010

Questions? 13 Gianni Pucciani, ISTQB course report 1 October 2010

Question 1 14 Which assertions about prioritizing test cases are true or false? A. To assign priorities to test cases according to the importance of the code components, I should ideally follow the programmer's recommendation. B. Assigning priorities means to distribute the allocated capacities equally in order to test each function at least once. C. The perception of the failure and its impact for the end user cannot be used for prioritizing test cases. D. Prioritising should only be used in emergency cases. E. Prioritising is also needed to make sure, that, in case of premature test termination, the best possible result was achieved. -----------------------------------------------1. A is true, B, C, D and E are false 2. E is true, A, B, C and D are false Gianni Pucciani, ISTQB course report 1 October 2010 3. B, C and D are true, A and E are false 4. All assertions are false.

Question 2 Which of the following test types can not be used in the stated test level? A. functionality in acceptance testing B. test on user acceptance in integration testing C. performance in component testing D. robustness in integration testing 15 Gianni Pucciani, ISTQB course report 1 October 2010

Question 3 16 The control graph of a function consists of 26 edges and 6 nodes. What are your considerations about this function with respect to testability? A. As long as the cyclomatic number of a function does not exceed this magnitude, testing is not a problem. B. The cyclomatic number is very high, which means that the function cannot be tested. C. Nothing can be said about the testability of these function. D. The high number of edges, compared to the number of nodes, indicates that the function contains many branches. The required testing effort, depending on the structure of the control graph, is definitively high Gianni Pucciani, ISTQB course report 1 October 2010 or even very high.

Question 4 Which of the statements are true, and which are false? A. In functional testing the internal control flow structure is used as basis for test case development. B. Structural testing is an approach for testing where the test cases are developed based on specifications (program or component specification). C. In functional testing the system is considered as a „black box“. Its behavior can only be verified by analyzing the input and the corresponding generated output values. D. Functional testing concerns functionality not implementation. E. Within functional testing test data is based on knowledge about the implementation. -----------------------------------------------1. A and D are true, B, C and E are false 2. C and D are true, A, B and E are false 3. A, B and E are true, C and D are false 4. B, C and D are true, A and E are false 17 Gianni Pucciani, ISTQB course report 1 October 2010

Question 5 18 During the acceptance test you find three faults. As identified later, one fault concerns the requirements, one fault is in the system design and one fault is in the source code. Which of the following statements normally applies to this situation? 1. The fault in the requirements will cause the lowest failure correction costs. 2. The fault in the source code will cause the highest failure correction costs. 3. The fault in the requirements will cause the highest failure correction costs. 4. The fault which is the simplest to correct, causes the highest error costs. Gianni Pucciani, ISTQB course report 1 October 2010

Question 6 19 You have successfully introduced test robots in your test team. That’s why you and your team are planned to be assigned to a new project. Four different projects apply for your help. Which one would you select? 1. Project A is a program that works in batch mode. The command line is the user interface for data input. 2. Project B is a customer specific program with a graphical user interface. Since the customer expects a ready-to-use product in a very short time-frame, the time pressure is enormous. 3. Project C is a new, company-owned and company strategic product with a graphical user interface. Gianni Pucciani, ISTQB course report 1 October 2010 4. Project D is a database that has been designed for

Fundamental Test Process (1) Test level acceptance testing Test level. . . . Test level component testing Planning Analysis & design Implementation & execution Evaluating exit criteria & reporting Test closure activities Test cycle 1. . . n and control Slide 20 • CTF 01. pptx • Stephan Hostie • 27/02/2021

Overview of Test Types � Four basic test types are differentiated Test types Functional testing Non-functional testing Structural testing Change related testing Requirements based Load testing Confirmation testing Business-process based Performance testing Regression testing Use-case based Volume/mass testing Security test Stress testing etc. Slide 21 • CTF 02. pptx • Stephan Hostie • 27/02/2021

Costs of Reviews Maximum extent Documents 50 pages Code 20 pages Number of consultants Individual preparation (relative) 5 10 pages/h 3 5 pages/h Review preparation Review meeting Review cost 25 hours 14 hours 5 person days 12 hours 10 hours 3 person days Development cost (relative) Development cost 2 pages/day 25 person days 1 pages/day 20 person days Review vs. development cost 20 % 15 % Savings (cost for review included) 14 – 25 % Reduction of failure cost up to 75 % Slide 22 • CTF 03. pptx • Stephan Hostie • 27/02/2021 (Bush, 90) (Frühauf, 91)

Test Design Techniques � Overview of different test design techniques: Test design techniques Systematic techniques Specification-based (black-box) techniques Equivalence class partitioning Boundary value analysis Decision table testing State transition testing Use case testing Slide 23 • CTF 04. pptx • Stephan Hostie • 27/02/2021 Non-systematic techniques Structure-based (white-box) techniques Control flow-based techniques Statement coverage Branch coverage Experience-based techniques Error guessing Exploratory testing

Test preparation Max Review Slide 24 • CTF 05. pptx • Stephan Hostie • 27/02/2021 io pe ra t O Sy st em en t po n om C Time of failure detection ns te st st Min te ui ld /b g od in C R sp equ ec ire ifi m ca e tio nts n Sy st em de si gn Relative failure repair cost Early Failure Detection as a Goal

Test and Configuration Management (1) Project manager Customer Re qu ire ct roje me nts P Configuration management Test tools SW n ig s nt ne po Re ela que de bor st to sc ate De rip sig tio on d nu n es ign pd ate s Requests to elaborate on requirements description Slide 25 • CTF 05. pptx • Stephan Hostie • 27/02/2021 Testware Software architect Test objects n ig es s De om m re i qu e R -C ts -D SW en n pla Testing Developer ms le rob Software engineer n P tio ma for in ult Fa Requests for tools