Review for Final Fall 2019 Jeff Offutt SWE

Review for Final – Fall 2019 Jeff Offutt SWE 437 Software Testing & Maintenance www. cs. gmu. edu/~offutt/

Parameters • 10: 30 – 1: 15 • Same room • 17 December, next Tuesday – LAST quiz retake 9 -December, 10: 00 am-noon, Engineering 4201 • Closed book / closed notes / closed neighbor – One sheet of definitions nd terms allowed – Hand-written, not typed, double-sided is okay • Comprehensive • You may take breaks (not all together) 437 - Final Review © Jeff Offutt, 2019 2

Summary: Software Maintenance • Week 2: Overview & evolution – Two essays 1. Overview of software maintenance and evolution 2. How the web resuscitated evolutionary design • Week 3: Coding and design for change – Two essays 1. Coding for change 2. Designing for change 437 - Final Review © Jeff Offutt, 2019 3

Summary: Part 1—Foundations 1. Why Test Software? (Ch 1) • Fault, error, failure, program state • Test maturity levels • Cost of late testing 2. Model-Driven Test Design (Ch 2) • RIPR model • Other terms 3. Test Automation (Ch 3) • Junit • Components of a test case 4. Developer Testing (Ch 4) • Agile and TDD 5. Criteria-Based Test Design (Ch 5) • Test requirements and criteria 437 - Final Review © Jeff Offutt, 2019 4

Cost of Late Testing 60 50 Assume $1000 unit cost, per fault, 100 faults K 0 $25 $3 60 40 30 20 Fault detection (%) 0 K Unit cost (X) 0 ir u q e R Fault origin (%) K K ts n e m e $10 0 K $1 3 $6 10 $2 K De n g si Pr og t / i n U st e T t r i t a In g e t on s e T em t s Sy s e T m y o t en l p De - t s o P t Software Engineering Institute; Carnegie Mellon University; Handbook CMU/SEI-96 -HB-002 437 - Final Review © Jeff Offutt, 2019 5

RIPR Model Test • Reachability • Infection • Propagation • Revealability 437 - Final Review Reaches Fault Infects Incorrect Program State © Jeff Offutt, 2019 Final Program State Observed Final Program State Incorrect Final State Propagate s Reveals Test Oracles 6

Model-Driven Test Design–Activities model / structure refined requirements / test specs test requirements Test Design software artifact IMPLEMENTATION ABSTRACTION LEVEL Test Automation pass / fail Test Evaluation 437 - Final Review DESIGN ABSTRACTION LEVEL test results Test Execution test scripts © Jeff Offutt, 2019 input values test cases 7

Criteria Based on Structures : Four ways to model software 1. Input Domain Characterization (sets) A: {0, 1, >1} B: {600, 700, 800} C: {swe, cs, isa, infs} 2. Graphs 3. Logical Expressions (not X or not Y) and A and B if (x > y) z = x - y; else z = 2 * x; 4. Syntactic Structures (grammars) 437 - Final Review © Jeff Offutt, 2019 8

Summary: Part 2 Coverage Criteria (1 of 4) • Input Space Partitioning – Ch 6 – Input Domain Modeling (6. 1) – Combination Strategies Criteria (6. 2) – Skipped 6. 3 & 6. 4 – not on final exam – Be sure to review the coverage definitions in 6. 2 – Know how to construct partitions and blocks – Know how to choose values 437 - Final Review © Jeff Offutt, 2019 9

ISP Coverage Criteria Subsumption All Combinations Coverage ACo. C T-Wise Coverage TWC Multiple Base Choice Coverage MBCC Pair-Wise Coverage PWC Base Choice Coverage BCC Each Choice Coverage ECC 437 - Final Review © Jeff Offutt, 2019 10

Summary: Part 2 Coverage Criteria (2 of 4) • Graph Coverage – Ch 7 – Overview (7. 1) – Graph Coverage Criteria (7. 2) – Graph Coverage for Source Code (7. 3) We skipped data flow—not on the final – Skipped 7. 4, 7. 5, 7. 6 – not on final exam – Be sure to review the definitions in 7. 2 – Know how to generate graphs for various artifacts and apply the criteria to them – Know how to choose values (EC, EPC, PPC, …) 437 - Final Review © Jeff Offutt, 2019 11

Graph Coverage Criteria Subsumption Complete Path Coverage CPC All-DU-Paths Coverage ADUP All-uses Coverage AUC All-defs Coverage ADC 437 - Final Review Prime Path Coverage PPC Edge-Pair Coverage EPC Edge Coverage EC Node Coverage NC © Jeff Offutt, 2019 Complete Round Trip Coverage CRTC Simple Round Trip Coverage SRTC 12

Summary: Part 2 Coverage Criteria (3 of 4) • Logic Coverage – Ch 8 – Logic Coverage Criteria (Active) (8. 1) – Structural Logic Coverage of Programs (8. 3) – Skipped 8. 2, 8. 4. 8. 5 – not on final exam – Be sure to review the coverage definitions in 8. 1 – Know how to apply the coverage criteria to programs, specs, and FSMs – Know how to choose values for CACC 437 - Final Review © Jeff Offutt, 2019 13

Logic Criteria Subsumption Combinatorial Clause Coverage COC Restricted Inactive Clause Coverage RICC Restricted Active Clause Coverage RACC Correlated Active Clause Coverage CACC General Inactive Clause Coverage GICC General Active Clause Coverage GACC Clause Coverage CC 437 - Final Review © Jeff Offutt, 2019 Predicate Coverage PC 14

Summary: Part 2 Coverage Criteria (4 of 4) • Syntax-based Testing – Ch 9 – Syntax-based Coverage Criteria (9. 1) – Input Space Grammars (9. 5) – Skipped 9. 2, 9. 3, 9. 4 – not on final exam – Understand how to derive tests from BNF – Assignment 10 is a good guide 437 - Final Review © Jeff Offutt, 2019 15

Representing Input Domains goal domain specified domain This region is a rich source of software errors … implemented domain 437 - Final Review © Jeff Offutt, 2019 16

Preparing & Format • Re-read the text and handouts • Study the homework assignments • Redo the quizzes • Try additional exercises in text • Get a good night’s sleep • Possibly different kinds of questions – Mostly problem solving—similar to quizzes, perhaps bigger – Short answer – Maybe a few T / F, M / C, single answer questions 437 - Final Review © Jeff Offutt, 2019 17

Posting Grades I will post on Blackboard: – Last assignment grades – Last quiz grades – Average assignment grades – Participation grades – Final exam score 437 - Final Review © Jeff Offutt, 2019 18

Final Grades • 15% participation, 30% quizzes, 20% assignments, 35% final exam • Unbiased grading – compute and sort: • • >= 98 : A+ > 93 : A > 90 : A> 86 : B+ > 80 : B > 75 : B> 65 : C – Probably one A+ per class • I will “slide the line” down to an appropriate break point 437 - Final Review © Jeff Offutt, 2019 19

What I Want You to Take Away You in 5 years Me Forgot everything about testing and high quality software Remember testing, but develop software the “same old way” Use these concepts to hand-design more effective tests for your software Apply automation and criteria when possible Convince your company to develop a quality mindset 437 - Final Review © Jeff Offutt, 2019 20