Review for Final Fall 2018 Jeff Offutt SWE

Review for Final – Fall 2018 Jeff Offutt SWE 637 Software Testing www. cs. gmu. edu/~offutt/

Parameters and logistics • 4: 30 – 7: 15 • Same room • 17 December, next Monday – 10 -December is a reading day – No quiz retakes after 10 -December • Closed book / closed notes / closed neighbor – One sheet of definitions allowed – Hand-written, not typed, double-sided is okay • Comprehensive • You may take breaks (not all together) • Spread out – separate tables and one table person 637 - Final Review © Jeff Offutt 2

Summary: Part 1—Foundations 1. Why Test Software? (Ch 1) • Fault, error, failure, program state • Test maturity levels • Cost of late testing 2. Model-Driven Test Design (Ch 2) • RIPR model • Other terms 3. Test Automation (Ch 3) • Junit • Components of a test case 4. Developer Testing (Ch 4) • Agile and TDD 5. Criteria-Based Test Design (Ch 5) • Test requirements and criteria 637 - Final Review © Jeff Offutt 3

Cost of Late Testing 60 Assume $1000 unit cost, per fault, 100 faults 50 $3 K 0 $25 60 40 30 20 $10 0 K $1 3 $6 10 $2 K Fault origin (%) Fault detection (%) 0 K K K Unit cost (X) 0 m ri e qu Re ts n e De is gn P / g ro t t i n U s Te t In t ra g e n it o s e T s Sy em s e T t t n e m y o l t - t s o p e D P Software Engineering Institute; Carnegie Mellon University; Handbook CMU/SEI-96 HB-002 637 - Final Review © Jeff Offutt 4

RIPR Model Test • Reachability Reaches • Infection • Propagation • Revealabilit y 637 - Final Review Fault Infect s Incorrec t Program State © Jeff Offutt Final Program State Observed Final Program State Incorrect Final State Propagate s Reveals Test Oracles 5

Model-Driven Test Design–Activities model / structure refined requirements / test specs test requirements Test Design software artifact IMPLEMENTATION ABSTRACTION LEVEL Test Automation pass / fail Test Evaluation 637 - Final Review DESIGN ABSTRACTION LEVEL test results Test Execution test scripts © Jeff Offutt input values test cases 6

Criteria Based on Structures : Four ways to model software 1. Input Domain Characterization (sets) A: {0, 1, >1} B: {600, 700, 800} C: {swe, cs, isa, infs} 2. Graphs 3. Logical Expressions (not X or not Y) and A and B 4. Syntactic Structures (grammars) if (x > y) z = x - y; else z = 2 * x; 637 - Final Review © Jeff Offutt 7

Summary: Part 2 Coverage Criteria (1 of 4) • Input Space Partitioning – Ch 6 – Input Domain Modeling (6. 1) – Combination Strategies Criteria (6. 2) – Handling Constraints Among Characteristics (6. 3) – Extended Example: Deriving an IDM from Java. Doc (6. 4) – Be sure to review the coverage definitions in 6. 2 – Know how to construct partitions and blocks – Know how to choose values 637 - Final Review © Jeff Offutt 8

ISP Coverage Criteria Subsumption All Combinations Coverage ACo. C T-Wise Coverage TWC Multiple Base Choice Coverage MBCC Pair-Wise Coverage PWC Base Choice Coverage BCC Each Choice Coverage ECC 637 - Final Review © Jeff Offutt 9

Summary: Part 2 Coverage Criteria (2 of 4) • Graph Coverage – Ch 7 – Overview (7. 1) – Graph Coverage Criteria (7. 2) – Graph Coverage for Source Code (7. 3) – Graph Coverage for Design Elements (7. 4) – Graph Coverage for Specifications (7. 5) – Graph Coverage for Use Cases (7. 6) – Be sure to review the definitions in 7. 2 – Know how to generate graphs for various artifacts and apply the criteria to them – Know how to choose values (EC, EPC, PPC, …) 637 - Final Review © Jeff Offutt 10

Graph Coverage Criteria Subsumption Complete Path Coverage CPC All-DUPaths Coverage ADUP All-uses Coverage AUC All-defs Coverage ADC 637 - Final Review Prime Path Coverage PPC Edge-Pair Coverage EPC Edge Coverage EC Node Coverage NC © Jeff Offutt Complete Round Trip Coverage CRTC Simple Round Trip Coverage SRTC 11

Summary: Part 2 Coverage Criteria (3 of 4) • Logic Coverage – Ch 8 – Semantic Logic Coverage Criteria (Active) (8. 1) – Syntactic Logic Coverage Criteria (Inactive) (8. 2) – Structural Logic Coverage of Programs (8. 3) – Specification-based Logic Coverage (8. 4) – Logic Coverage of FSMs (8. 5) – Be sure to review the coverage definitions in 8. 1 – Know how to apply the coverage criteria to programs, specs, and FSMs – Know how to choose values (CACC, …) 637 - Final Review © Jeff Offutt 12

Logic Criteria Subsumption Combinatorial Clause Coverage COC Restricted Inactive Clause Coverage RICC Restricted Active Clause Coverage RACC Correlated Active Clause Coverage CACC General Inactive Clause Coverage GICC General Active Clause Coverage GACC Clause Coverage CC 637 - Final Review © Jeff Offutt Predicat e Coverag e PC 13

Summary: Part 2 Coverage Criteria (4 of 4) • Syntax-based Testing – Ch 9 – Syntax-based Coverage Criteria (9. 1) – Program-based Grammars (9. 2) – Integration and Object-Oriented Testing (9. 3) • Concepts, not all the details – Specification-based Grammars (09. 4) – Input Space Grammars (9. 5) – Understand how to derive tests from BNF – Understand mutation analysis and mutation testing 637 - Final Review © Jeff Offutt 14

Representing Input Domains goal domain specified domain This region is a rich source of software errors … implemented domain 637 - Final Review © Jeff Offutt 15

Preparing & Format • Re-read the text and handouts • Study the homework assignments • Redo the quizzes • Try exercises in text • Get a good night’s sleep • Possibly different kinds of questions – Mostly problem solving—similar to quizzes, perhaps bigger – Short answer – Maybe a few T / F, M / C, single answer questions 637 - Final Review © Jeff Offutt 16

Posting Grades I will post on Blackboard: – Quiz 13 grades – Assignment 12 grades – Average quiz grades – Average assignment grades – Participation grades – Final exam score 637 - Final Review © Jeff Offutt 17

Final Grades • 15% participation, 20% assignments, 30% quizzes, 35% final exam • Unbiased grading – compute and sort: • >= 98 : A+ • > 93 : A • > 90 : A • > 86 : B+ • > 80 : B • > 75 : B • > 65 : C – Probably one A+ per class • I will “slide the line” down to an appropriate “break point” 637 - Final Review © Jeff Offutt 18

What I Want You to Take Away You in 5 years Me Forgot everything about testing and high quality software Remember testing, but develop software the “same old way” Use these concepts to hand-design more effective tests for your software Apply automation and criteria when possible Convince your company to develop a quality mindset 637 - Final Review © Jeff Offutt 19