Software Verification and Validation Lecture No 13 Module

Software Verification and Validation Lecture No. 13

Module 75: Introduction to Software Verification

Introduction to Software Verification § Software development lifecycle models and processes § Waterfall model § Spiral model § Rapid Application Development (RAD) § Evolutionary prototyping § Etc. § Each stage can potentially contain issues such as: § Misconceptions § Misunderstandings § Errors…

Introduction to Software Verification § We need to spend time and financial resources to fix these errors or bugs in the following manner (ref: textbook): § Cost to fix a bug increases exponentially (10 x) § i. e. , it increases tenfold as time increases § E. g. , a bug found during specification costs $1 to fix. § … if found in design cost is $10 § … if found in code cost is $100 § … if found in released software cost is $1000

Module 76: Verification Methods

Verification Methods § Defect prevention: Error blocking and error source removal. § Defect removal: § Inspection - this lecture. § Testing, etc. § Defect containment: Fault tolerance and failure containment (safety assurance).

Verification Methods § Verification is process of assessing a software product or system during a particular phase § to determine if it meets requirements or conditions § that were specified at the beginning of that phase. § It is a static process and it includes substantiation of artifacts such as § SRS, SDS, § design and § Code / program.

Verification Methods § We mainly divide verification methods into § Peer Reviews § Walkthroughs § Inspections § We make use of checklists which represent a set of timetested means of making sure nothing is forgotten in any process is a checklist § A verification checklist will help manage all of the details that must be attended to in an artifact.

Verification Methods § From informal to formal Informal Walkthrough Peer Review Inspection

Verification Methods Method Family Walkthroughs Typical Goals Typical Attributes Minimal overhead Little/no preparation Informal process No measurement Not Formal Tech. Review Developer training Quick turnaround Technical Reviews Requirements elicitation Ambiguity resolution Training Formal process Author presentation Wide range of discussion Inspections Detect and remove all defects efficiently and effectively. Formal process Checklists, Measurements Verify phase

Verification Methods § Applications § SRS verification § Design verification § Code verification §… § Each step in adopted application development lifecycle requires verification before an artifact becomes a milestone and goes to the next stage.

Module 77: Walkthrough

Walkthrough § Author of the artifact leads the review process and the team members ask questions and spot possible errors against development standards and other issues. § The meeting is usually led by author of the document under review and attended by other members of the team. § Review sessions may be formal or informal. § Before walkthrough meeting, preparation by reviewers and then, optionally, a formal review report containing list of findings.

Walkthrough § Defects/issues are noted down so that they can be fixed and they may be tracked to closure. § Main purpose of walkthrough is to enable learning about the content of the document under review to help team members gain an understanding of the content of the document and also to find defects § Quality of results depend upon personal brilliance of the reviewer

Module 78: Peer Review

Peer Review § Peer Reviews are documented and use a defect detection process that has peers and technical specialist as part of review process. § Review process doesn't involve management participation. § It is usually led by trained moderator who is NOT the author. § The report is prepared with the list of issues that needs to be addressed.

Peer Review § Peer review requires authors to present their work, § Involves a team of 2 to 7 professionals § Report optional to author and it presents issues in the shape a few faults. § Question arises: what are types of issues. § Issues / faults present those situations that may impact desired results

Peer Review § Example–issue Classification § Severe § Defects that may cause incorrect results or behavior § Moderate § Defects that may affect limited areas of functionality that can either be worked around or ignored. § Minor § Defects that can be overlooked with no loss of functionality.

Module 79: Inspections for Verification

Inspection for Verification § M. E. Fagan of IBM defined inspections in 1976 § Inspection is led by a trained moderator, who is not the author. Inspection is most formal and driven by checklists and rules. § This review process makes use of entry and exit criteria.

Inspection for Verification § Inspections requires formal § presenter in the shape of moderator(NOT author), § prior preparation of complete team, § involvement of a team of 3 to 6 members, § formal report by moderator, § requires skilled staff and is expansive.

Inspection for Verification § Inspections can find about 60% of all defects. § Inspections are technical, not management. § Inspections results can assess/improve quality of: § work product § software development process § review process

Inspection for Verification § Inspections reduce total project cost, but have their own execution costs § Upstream defect removal is 10 -100 times cheaper. § Reviews disseminate domain knowledge, development skills, and corporate culture

Module 80: Fagan Inspections for Verification

Fagan Inspections for Verification § We share steps involved in Fagan Inspections: 1. Planning 2. Overview (1 -to-n meeting) 3. Preparation (individual inspection) 4. Inspection (n-to-n meeting) 5. Rework 6. Follow-up

Fagan Inspections for Verification 1. Planning § Entry criteria: what to inspect § Team size: about 4 persons § Developers/testers from similar projects § Inspectors are not authors 2. Overview § Author-inspectors meeting § General background information § functional/structural/inf o. , intentions

Fagan Inspections for Verification § Assign individual tasks: § coverage of important areas § moderate overlap 3. Preparation or individual inspection § Independent analysis/examination § Code as well as other document § Individual results: § questions/guesses § potential defects

Fagan Inspections for Verification 4. Inspection (generic: collection) § Meeting to collect / consolidate individual inspection results § Team leader/meeting moderator (1) § Reader/presenter: summarize/paraphrase for individual pieces § Defect identification, but not solutions § No more than 2 hours § Inspection report

Fagan Inspections for Verification 5. Rework § Author's response § Defect fixing (solutions) 6. Follow-up § Resolution verification by moderator § Re-inspection? § Fagan inspection in practice § Widely used in industry § Evaluation studies § Variations and other inspections

Module 81: Examples of Checklists for SRS document Inspections

Examples of Checklists for SRS Inspections § Importance of checklists § Represent industrial best practices as templates § Provide a means of use of preexisting knowledge base and lessons learnt § Help conduct inspections / audits in an efficient and appropriate manner § Artifact specific checklists § We discuss example checklists for § SRS, Code… § We have SRS and checklist as input to inspection

Examples of Checklists for SRS Inspections § General (SRS requirements) Checklist § functional overview of system provided? § software and hardware environments been specified? § assumptions that affect implementation stated? § Has every definition, acronym and abbreviation been defined in glossary, if they are correct? § Are requirements, interfaces, constraints, definitions, etc. listed in appropriate sections?

Examples of Checklists for SRS Inspections § Functional Requirements Checklist § Have all requirements communicated by customer been specified? § Are all inputs to a function sufficient to perform the required function? § Are undesired events / inputs considered and their required responses specified? § Are all use case descriptions provided? § Are all misuse cases documented?

Examples of Checklists for SRS Inspections § Interface Checklist §… § Non-functional Requirements Checklist §… § Requirements Quality §… §…

Examples of Checklists for SRS Inspections § By and large, checklist for requirements verification must check if the document is: § Complete, consistent, correct § Precise, unambiguous, clear § Relevant, testable, understandable § Traceable, feasible, prioritized § Free from design details

Module 82: Examples of Checklists for Code Inspections

Examples of Checklists for Code Inspections § Code checklists are language specific § Example code inspection checklist: § Code formatting: § Are proper alignments followed (left margin) § Are code block starting point/ending point easily identifiable. § Are naming conventions (Pascal, Camel. Case, Hungarian notation followed.

Examples of Checklists for Code Inspections § Coding best practices: § Are all similar values grouped under enumerated data types § Are comments properly given § Are comments provided for what is done rather why is done § Are there nested if/else statement in the code? § Is there a separation of concerns through implementation of any framework §…

Examples of Checklists for Code Inspections § Structure §… § Variables §… § Comments §… § Loop and branches §… § General §…

Module 83: Examples of Checklists for User Documentation Inspections

Checklists for User Documentation Inspections § Example of documentation review checklist: § Does title page include required information § The purpose of document is clear and complete. § All known audiences / customers / users are described thoroughly and accurately. § Scope of document is accurate and complete. § Product version numbers / release dates are accurate. § Table of contents clear

Checklists for User Documentation Inspections § All steps in the procedure accurate and complete. § All screen shots are accurate and complete. § All supporting texts are accurate and complete. § All corresponding screen shots accurately display current version of software and clearly relate to the step text. § All charts, graphs, and diagrams are labeled accurately and consistently. §…

Module 84: Code Review Example

Code Review Example § We need § Moderator § Team members § Authors § Checklists § Reporting forms § Artifact for inspection/review (Code) § Let author presents code and moderator conducts § Planning, Overview, Preparation and inspection with two team members

Code Review Example § Checklist § § Names are simple and if possible § short and are spelt correctly § Names contain units where applicable § Enums are used instead of int constants where applicable § All class, variable, and method § § modifiers are correct. There is no commented out code § § There is no dead code § Debugging code is absent § Follows coding conventions § § § § No stack traces are printed Variables not used with null values Code is not repeated or duplicated There is an else block for every if clause even if it is empty No complex, negatively named or long Boolean expressions No empty blocks of code Constructors don’t accept null values Arrays checked for out of bound Catch clauses are appropriately used Exceptions not ignored, if caught. Files/Sockets/Cursors and other resources are properly closed.

Code Review Example

Code Review Example § Issues Classification § Severe § Defects that may cause incorrect results or behavior § Moderate § Defects that may affect limited areas of functionality that can either be worked around or ignored. § Minor § Defects that can be overlooked with no loss of functionality.

Code Review Example § Example Issue Log Form

Code Review Example § Issues found § Severe § Try-catch block not implemented § Moderate § Constructor not present § Missing class, function and variable comments. § Minor § No naming convention followed e. g. , Hungarian notation is not followed for variable names, function names etc.

Code Review Example § Next steps: § Review meeting § Consolidated and comprehensive listing of issues prepared. § Shared with key stakeholders § Authors are given time to rework § Moderator (or representative) verifies if the corrections are made.