ECE 453 CS 447 SE 465 Software Testing

ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance Instructor Kostas Kontogiannis 1

Welcome • Welcome to the Software Testing and Quality Assurance course • Demanding, challenging and, rewarding course • A glimpse of what software testing in theory and practice is all about 2

How to reach me • Kostas Kontogiannis – Web: www. swen. uwaterloo. ca/~kostas – E-mail: kostas@swen. uwaterloo. ca – Tel. (ext. 2840) – Office hours: • Monday 17: 30 – 18: 30 • DC 2539 3

Software Testing CHAPTER 1 Unit 1 4

Outline for today èIntroduction • Course description • Software Testing, Quality Assurance, and Evolution basics 5

Course components • 3 lectures – SE 465 MC 4045 • 14: 30 -15: 20 MWF – ECE 453 / CS 447 RCH 301 • 16: 30 – 17: 20 MWF • 3 tutorials – SE 465 MC 4045 • 15: 30 – 16: 20 MW – ECE 453 / CS 447 RCH 305 • 8: 00 – 8: 50 W • 1 project – Evolution – Testing 6

Course website • • http: //www. swen. uwaterloo. ca/~kostas/ECE 453 -06/ schedule lecture notes & slides recommended book – Software Testing A Craftsman's Approach 2 nd edition, Paul C. Jorgensen, CRC Press • • past exams project description grade allocation course news system 7

Project component • Design and implement software for an IP Telephony application • Groups of 4 • 30% of total grade • Two parts: – System Enhancements • Requirements • Design • Implementation – Testing • Black Box • White Box 8

Goals of this course • Become familiar with software testing concepts • Learn how to design and plan software testing activities • Experience software testing in a project that uses several components è Note: – “Scratching the surface of software testing and QA” – “Fitting you to become an amateur software tester” 9

Course outline Unit 1: Overview of Software Testing (Chapter 1 -2) • Unit 2: Mathematics for S/W Testers (Chapters 3 -4) • Unit 3: Black Box / Functional Testing (Chapters 5 -8) • Unit 4: White Box / Structural Testing (Chapters 9 -11) • Unit 5: Integration and System Testing (Chapter 12 -13) • Unit 6: Software Reliability and Quality Assurance Basics • Unit 7: Software Reengineering and Evolution Basics • Unit 8: Project Planning and Management 10

What to do as a first Task • Visit the Web site • Go to the Project section – Complete Part I • Administration – Complete Part II • Preparation 11

Outline for today • Introduction • Course description èSoftware testing basics 12

Software Lifecycle • • Requirements phase Analysis phase Design phase (System and Object) Implementation phase Testing phase Integration phase Maintenance phase Retirement 13

Software Lifecycle Activities. . . and their models Requirements Elicitation Analysis Expressed in Terms Of System Design Structured By Object Design Implementation Implemented By Realized By Verified By class. . . Use Case Model Application Domain Objects Subsystems Testing Solution Domain Objects Source Code ? class. . ? Test Cases 14

Software Testing : Definition “The process of devising a set of inputs to a given piece of software that will cause the software to exercise some portion of its code. The developer of the software can then check that the results produced by the software in accord with his or her expectations. ” www. reactive-systems. com/esda-glossary. msp 15

Software Testing : Definition “Software testing is a process used to identify the correctness, completeness and quality of developed computer software. Actually, testing can never establish the correctness of computer software, as this can only be done by formal verification (and only when there is no mistake in the formal verification process). It can only find defects, not prove that there are none. ” en. wikipedia. org/wiki/Software_testing 16

A Testing Cycle • Although testing varies between organizations, there is a cycle to testing: 1. Requirements Analysis: Testing should begin in the requirements phase of the software life cycle (SDLC). 2. Design Analysis: During the design phase, testers work with developers in determining what aspects of a design are testable and under what parameter those testers work. 3. Test Planning: Test Strategy, Test Plan(s), Test Bed creation. 4. Test Development: Test Procedures, Test Scenarios, Test Cases, Test Scripts to use in testing software. 5. Test Execution: Testers execute the software based on the plans and tests and report any errors found to the development team. 6. Test Reporting: Once testing is completed, testers generate metrics and make final reports on their test effort and whether or not the software tested is ready for release. 7. Retesting the Defects 17

Some Software Testing Objectives • • Find as many defects as possible. Find important problems fast. Assess perceived quality risks. Advise about perceived project risks. Advise about perceived quality. Certify to a given standard. Assess conformance to a specification (requirements, design, or product claims). 18

Software Testing in RUP Phases Process Workflows Inception Elaboration Construction Transition Business Modeling Requirements Architecture & Design Implementation Test Deployment Supporting Workflows Configuration Mgmt Management Environment Preliminary Iteration(s) Iter. #1 Iter. #2 Iter. #n+1 #n+2 Iterations Iter. #m+1 19

Software Testing in RUP • RUP Testing Philosophy – – Iterative development. Low up-front documentation Holistic approach. Automation • Actors and Roles – – Test Manager Test Analyst Test Designer Tester 20

Software Testing in XP • The biggest difference between XP projects and most ‘traditional’ software development projects is the concept of test-driven development. • With XP, every chunk of code is covered by unit tests, which must all pass all the time. The absence of unit-level and regression bugs means that testers actually get to focus on their job: making sure the code does what the customer wanted. • The acceptance tests define the level of quality the customer has specified (and paid for!) 21

Software Testing in XP Characteristics Planning Every 2 -3 weeks Write tests Release Pair Programming + Refactoring Test Integration Min. daily • Evolutionary development • Collection of “ 12 Best Practices“ • Focus on working code that implements customer needs (rather than documents) • Testing is a crucial element of the process • Focus on flexibility and efficiency of the process • Designed for small teams (<10) 22

Software Testing in XP • Activities testers perform on XP teams. – Negotiate quality with the customer (it’s not YOUR standard of quality, it’s what the customer desires and is willing to pay for!) – Clarify stories, flush out hidden assumptions – Enable accurate estimates for both programming and testing tasks – Make sure the acceptance tests verify the quality specified by the customer – Help the team automate tests – Help the team produce testable code – Form an integral part of the continuous feedback loop that keeps the team on track 23

Software Quality Assurance : Definition “The function of software quality that assures that the standards, processes, and procedures are appropriate for the project and are correctly implemented. ” www. hq. nasa. gov/office/codeq/software/umbrella_defs. htm 24

Software process improvement initiatives • Capability maturity model (CMM) – http: //www. sei. cmu. edu/cmms/cmms. html • ISO 9000 -series • ISO/IEC 15504 – Standard for Software Process Assessment (SPICE) – http: //www-sqi. cit. gu. edu. au/spice/ 25 ©Steven Schach 2002 [modified]

Capability Maturity Model • A set of strategies for improving the software process – – – SW–CMM for software P–CMM for human resources (“people”) SE–CMM for systems engineering IPD–CMM for integrated product development SA–CMM for software acquisition • These strategies are being unified into CMMI (capability maturity model integration) • Developed by Software Engineering Institute (SEI) ©Steven Schach 2002 [modified] 26

SW–CMM • A strategy for improving the software process • Put forward in 1986 by the SEI • Fundamental ideas: – Improving the software process leads to • Improved software quality • Delivery on time, within budget – Improved management leads to • Improved techniques • Five levels of “maturity” are defined • Organization advances stepwise from level to level 27 ©Steven Schach 2002

SW–CMM Summary 28 ©Steven Schach 2002

Software Evolution : Definition “A continuous change from a lesser, simpler, or worse state to a higher or better state [for a software system]” www. cc. gatech. edu/reverse/glossary. htm 29

Software Maintenance : Definition “Software maintenance is one of the activities in software engineering, and is the process of enhancing, porting, and optimizing deployed software, as well as remedying defects” en. wikipedia. org/wiki/Software_maintenance 30

Average cost distribution (1976– 1981 data) Object-Oriented and Classical Software Engineer 5 th Edition, Schach (2002) 31

Summary • Testing is an important part of the Software Lifecycle • Highly technical and challenging • Is affected by the selected process • Quality Assurance is paramount both for mission critical and non-critical systems • Software Evolution aims to keep systems operational when environment changes occur 32