Software Qualities Maintainer Good Documentation Readable Code Good
Software Qualities Maintainer Good Documentation Readable Code Good Design Functionality Ease of use Ease of learning Reliability Correctness Efficiency Low Cost Portability Increased productivity Customer User CSE 470: Software Engineering: Quality Assurance 236
Software Quality Assurance l Use of analysis to validate artifacts n requirements, designs, code, test plans l Technical reviews l Document reviews l Compliance to standards l Control of changes CSE 470: Software Engineering: Quality Assurance 237
Costs of Poor Quality l Increased time to find and fix problems l Increased cost to distribute modifications l Increased customer support l Product liability l Failure in the market place CSE 470: Software Engineering: Quality Assurance 238
QA Terminology l Correctness l Fault l Reliability l Error l Testing l Verification l Debugging l Validation l Failure l V&V CSE 470: Software Engineering: Quality Assurance 239
Terminology l Correctness: artifact is consistent with its specification n l Specification could be wrong or incomplete Rarely is software known to be correct Rarely is the specification correct Reliability: probability that the software is correct n Statistical measure based on past performance u e. g. , mean time to failure CSE 470: Software Engineering: Quality Assurance 240
More terminology l Testing: entails executing the software on selected test cases n n n l Evaluate the results (oracle) Evaluate the performance Evaluate the ease of use Common Wisdom: Testing reveals bugs but does not guarantee the absence of bugs n n How should you select test cases? How do you know when to stop testing? CSE 470: Software Engineering: Quality Assurance 241
More terminology l l Failure: an erroneous result n incorrect outputs/response for given inputs/stimuli n fails to meet real-time constraints Error: incorrect concept n l may cause failures if not corrected Fault: the cause of one or more failures n discovered after release CSE 470: Software Engineering: Quality Assurance 242
More terminology l Debugging: the process of finding the cause of a “bug” and a way to fix it n l w/o introducing additional bugs! Verification: process of proving, using mathematical reasoning, that a program is “correct” is expensive and is not always possible n is not foolproof n CSE 470: Software Engineering: Quality Assurance 243
More terminology l Validation: process associated with showing that the software performs reasonably well l V & V: verification & validation? n more typically equated with validation CSE 470: Software Engineering: Quality Assurance 244
Many different kinds of testing l l Unit testing: test individual components n test stubs simulate called components n test harness simulates “outer” context and maintains stubs Integration testing: combine components and test them n l follows build plan System testing: test whole system CSE 470: Software Engineering: Quality Assurance 245
More kinds of testing Acceptance testing: testing to determine if the product is acceptable l Regression testing: retesting after the system has been modified l determine “old” test cases that must be reexecuted n determine what new test cases are required n CSE 470: Software Engineering: Quality Assurance 246
More kinds of testing l Black box / functional testing: n l White box / structural testing: n l testing based on specifications testing based on looking at the artifact Both black box and white box testing are needed CSE 470: Software Engineering: Quality Assurance 247
Testing is hard work l Typically 50% of software development effort goes into testing n l How to identify “good” test cases? n l up to 85% for life-critical software high probability of finding a new error Objective is to find errors n test case is “successful” if it finds a new error CSE 470: Software Engineering: Quality Assurance 248
Testing is hard work (cont. ) Psychologically difficult for a programmer to test his/her own code thoroughly l Exhaustive testing requires testing all combinations of input values l n Sorting an array of size 10 containing integers in the range 1. . 10 has 10! combinations (3, 628, 800 cases) CSE 470: Software Engineering: Quality Assurance 249
Testing l l CAN: n Uncover errors n Show specifications are met for specific test cases n Be an indication of overall reliability CANNOT: n Prove that a program is error-free CSE 470: Software Engineering: Quality Assurance 250
Testing Principles l Tests should be traceable to requirements l Tests should be planned long before testing begins l Exhaustive testing is not possible n 80% of all errors typically occur in 20% of the modules n test cases should be chosen to maximize likelihood of finding an error CSE 470: Software Engineering: Quality Assurance 251
Testing Principles (cont. ) l Testing should be done by someone other than the developers Developers do original testing n SQA does independent testing n u usually l black box testing Automated testing tools should be used Reduce testing costs n Reduce likelihood of human error n CSE 470: Software Engineering: Quality Assurance 252
Testability l Simple software is easier to test n l l Output is sufficient to determine correct behavior Performs its own tests for internal errors n l l l minimize coupling, maximize cohesion raises meaningful exceptions All code is reachable Independent modules can be tested in isolation Documentation is complete and accurate CSE 470: Software Engineering: Quality Assurance 253
Quality is an on-going concern l l l You can’t build quality into a system after the fact Quality should be a consideration during every phase of development Plan for testing / validation in all phases n n requirements -> functional test cases design -> functional and structural test cases code -> enhanced func & struc test cases maintenance -> further enhanced func & struc test cases CSE 470: Software Engineering: Quality Assurance 254
Debugging l Find the cause of a failure and fix it n l an art, not a science Debugging is difficult because symptom may appear long after the fault occurs n symptom may be difficult to reproduce n symptom may be intermittent n l Unit testing helps localize errors CSE 470: Software Engineering: Quality Assurance 255
Software Reviews Individuals read and comment on the software artifacts l Very human intensive l Overriding evidence shows that it l improves quality and productivity n reduces cost n l It is usually one of the first activities to be dropped when schedules get tight CSE 470: Software Engineering: Quality Assurance 256
Software Reviews (cont. ) l l Applicable to all software artifacts n code inspections n requirements and design reviews n walk-throughs Recent research shows that n particular kind of review, size of team, etc. doesn’t matter n need at least one good, dedicated person doing the review CSE 470: Software Engineering: Quality Assurance 257
Typical Review Team l l Developer -- presents the material Moderator -- keeps the review on track n l l makes sure everyone abides by the process Secretary -- takes minutes, documents problems found Optionally n n Apprentice -- learning about the project Domain expert -- familiar with the domain and can verify assumptions CSE 470: Software Engineering: Quality Assurance 258
Software Review Guidelines l Review the artifact n l l l don’t attack the developer Stick to an agenda Limit debate Identify problems, not solutions Keep accurate notes Establish and follow evaluation guidelines Limit number of participants CSE 470: Software Engineering: Quality Assurance 259
Technical Review Guidelines (cont. ) l Prepare beforehand n l Allocate resources for reviews n l both developers and reviewers people and time Possible outcomes n accept product as is n reject product outright n accept product provisionally CSE 470: Software Engineering: Quality Assurance 260
Sample evaluation Guidelines: Code Inspection l l l Has the design been correctly translated to code? Are the language features used appropriately? Are coding standards followed? Are documentation standards followed? Are there misspellings or typos? Are the comments accurate and unambiguous? CSE 470: Software Engineering: Quality Assurance 261
Sample evaluation Guidelines: Code Inspection (cont. ) l l l Are data types and declarations appropriate? Are all constants correct? Are all variables initialized before being used? Are there overly complex conditions? Is there unreachable code? Are there obvious inefficiencies? CSE 470: Software Engineering: Quality Assurance 262
SQA Summary U. S. software costs $200 billion/year l Need to l improve software quality n reduce costs n u V&V l is over 50% of the cost Improving V&V should reduce costs significantly while improving quality CSE 470: Software Engineering: Quality Assurance 263
- Slides: 28