Software Quality Assurance What is quality software and

  • Slides: 50
Download presentation
Software Quality Assurance “What is quality software and how to achieve it? ” TCS

Software Quality Assurance “What is quality software and how to achieve it? ” TCS 2411 Software Engineering 1

Lecture Objectives z. To discuss what is meant by quality in software z. To

Lecture Objectives z. To discuss what is meant by quality in software z. To understand the importance of quality in software engineering z. To describe the features of quality software z. To describe the measures used in assessing software quality z. To understand the international standards that are applied to assess software quality TCS 2411 Software Engineering 2

What Is Quality Software? z. Is it software that works? z. Is it software

What Is Quality Software? z. Is it software that works? z. Is it software that is error-free? z. Is it software that is very well-documented and has help facility for all its functions? z. Is it software that does not ‘hang’ the machine? z. Is it software that does not ‘bomb’? TCS 2411 Software Engineering 3

Group Project z. How do you ensure the quality of the software you are

Group Project z. How do you ensure the quality of the software you are developing? z. Have you defined the standards and tasks to ensure the quality of the software? z. Will the customer say that your software is a quality product? z. Are you fulfilling the customer’s expectations? TCS 2411 Software Engineering 4

Software Quality Definition Conformance to z. Explicitly stated functional and performance requirements z. Explicitly

Software Quality Definition Conformance to z. Explicitly stated functional and performance requirements z. Explicitly documented development standards, and z. Implicit characteristics that are expected of all professionally developed software (Pressman) TCS 2411 Software Engineering 5

Software Quality Factors z. Low defects level when deployed y. Zero defect most preferably

Software Quality Factors z. Low defects level when deployed y. Zero defect most preferably z. High reliability y. Capability of running without crashes z. Majority of the user satisfy with software when conducted the survey z. Effective customer support z. Rapid defect repair TCS 2411 Software Engineering 6

Software Qualities z z z z z Correctness Reliability Robustness Performance User friendliness Verifiability

Software Qualities z z z z z Correctness Reliability Robustness Performance User friendliness Verifiability Maintainability Reparability Safety z z z z Evolvability Reusability Portability Understandability Productivity Size Timeliness Visibility Select the critical attributes and plan how to achieve them TCS 2411 Software Engineering 7

Benefits Of Software Quality z. Reduced maintenance cost z. Stable and useful product z.

Benefits Of Software Quality z. Reduced maintenance cost z. Stable and useful product z. Satisfy customer needs z. Better chances for continuing releases z. Build corporate culture and identity z. Better chances for software and design reuse TCS 2411 Software Engineering 8

Software Quality Assurance z Quality assurance – a set of establish framework (procedures &

Software Quality Assurance z Quality assurance – a set of establish framework (procedures & standard) used by an organization to achieve high quality software z Essential activity for any business that produce product to used by others y It need to be planned and systematic y It does not just happen z Assure each of the software qualities is met y Goal set in requirements specification y Goal realized in the implementation z SQA is applied at every stage of the software process z Quality standard is selected to apply to the software process TCS 2411 Software Engineering 9

Why SQA Payoff ? cost to find and fix a defect 100 80 -130

Why SQA Payoff ? cost to find and fix a defect 100 80 -130 log scale 10 1 SQA pays off here because defects are cheap to fix 1. 00 1. 30 Reqmts Design TCS 2411 Software Engineering 2. 00 Code 13. 00 4. 00 Test System test Field use 10

Principle Of SQA 1. Set the standard and quality attributes that a software product

Principle Of SQA 1. Set the standard and quality attributes that a software product must meet y The goal to achieve 2. Measure the quality of software product y There is a way to determine how well the product conforms to the standards and quality attributes 3. Track the values of the quality attributes y It is possible to assess how well we are doing 4. Use the information of software quality to improve the quality of future software product y There is a feedback into the software development process TCS 2411 Software Engineering 11

SQA Activities z. Application of Technical Methods z. Conduct of Formal Technical Reviews z.

SQA Activities z. Application of Technical Methods z. Conduct of Formal Technical Reviews z. Software Testing z. Enforcement of Standards z. Control of Change z. Measurement z. Record keeping and Reporting TCS 2411 Software Engineering 12

Application of Technical Methods z Select appropriate tools and methods to capture system requirements,

Application of Technical Methods z Select appropriate tools and methods to capture system requirements, analyze system, design, implementation and testing Why use methods and tools y. To ensure high quality is achieve y. Analyst achieve high quality specifications y. Designer develop high quality designs y. Ability to measure quality in specifications and designs TCS 2411 Software Engineering 13

Conduct of Formal Technical Reviews z. FTR is an activity to assess quality z.

Conduct of Formal Technical Reviews z. FTR is an activity to assess quality z. A stylised meeting conducted by technical staff with the sole purpose of uncovering quality problems z. Found to be effective in uncovering defects in software z. All major software items should be subjected to technical review TCS 2411 Software Engineering 14

Software Testing z Testing is a process of executing a program with the intent

Software Testing z Testing is a process of executing a program with the intent of finding errors, locate errors and to proof system correctness z Testing is conducted based on the developed test cases y. Capture actual output y. Compare actual output with expected output x. Actual == Expected : Test case succeed x. Actual != Expected : Test case failed z All test cases result must be recorded TCS 2411 Software Engineering 15

Software Testing(Cont) z. To identify errors in software developed z. Test case design methods

Software Testing(Cont) z. To identify errors in software developed z. Test case design methods produces tests to be used on the software z. Development of test strategy for a set of tests that uncovers all possible errors z. Thorough testing is not as effective as expected in most cases TCS 2411 Software Engineering 16

Enforcement of Standards z. Formal standards and procedures varies from company to company z.

Enforcement of Standards z. Formal standards and procedures varies from company to company z. Could be dictated by customer, regulations, or self-imposed z. If formal (written) standards exist, they must be followed to ensure quality z. Assessment of compliance is done through FTR or audit TCS 2411 Software Engineering 17

Control of Change z An activity executed throughout the system life cycle to control

Control of Change z An activity executed throughout the system life cycle to control change of products and life cycle artifacts z Items that needs control changes y Plans y Specification y Procedures y Audit y Report y Support • Need to be identified and described • Need to be uniquely itemized • Need to be organized z A software library need to be construct that store, manages and track these items TCS 2411 Software Engineering 18

Control of Change (Cont) z. Every change has potential for introducing errors or creating

Control of Change (Cont) z. Every change has potential for introducing errors or creating side effects that propagate errors z. Change control process ensures quality z. Formalizing requests for change, evaluating nature of change, and controlling the impact of change TCS 2411 Software Engineering 19

Measurement z Measurement : Numeric derivation for some attribute of a software product /

Measurement z Measurement : Numeric derivation for some attribute of a software product / process y. Compare the value with standard apply y. Often expressed in $$$ and days y. Integral part to any engineering discipline z Software metrics must be collected to track quality z Also to assess the impact of methodological and procedural changes on improved software quality TCS 2411 Software Engineering 20

Record keeping and Reporting z. Historical record for project - results of reviews, audits,

Record keeping and Reporting z. Historical record for project - results of reviews, audits, change control, testing, other SQA activities z. Provide procedures for collection and dissemination of SQA information z. Dissemination to development staff based on need-to-know basis TCS 2411 Software Engineering 21

Mc. Call Software Quality Model Product Revision Product Transition Product Operations TCS 2411 Software

Mc. Call Software Quality Model Product Revision Product Transition Product Operations TCS 2411 Software Engineering 22

Mc. Call’s Software Quality Factors z. Product Operations y. Operational characteristics z. Product Revision

Mc. Call’s Software Quality Factors z. Product Operations y. Operational characteristics z. Product Revision y. Ability to undergo changes z. Product Transition y. Adaptability to new environments TCS 2411 Software Engineering 23

Mc. Call’s Software Quality Factors (Cont) Factor Product Revision Product Transition Criteria Description Maintainability

Mc. Call’s Software Quality Factors (Cont) Factor Product Revision Product Transition Criteria Description Maintainability Can I fix it? Flexibility Can I change it? Testability Can I test it? Portability Will I be able to use it on another machine? Reusability Will I be able to reuse some of the other software in other application? Interoperability Will I be able to interface it with another system? Product Operation Correctness Does it do what I want? Reliability Does it do it accurately all the time? Efficiency Will it run as well as it can? Integrity Is it secure? Usability TCS 2411 Software Engineering Is it easy to use? 24

FCM By Mc. Call Approach Factors – user concern oriented Reliability Criteria – software

FCM By Mc. Call Approach Factors – user concern oriented Reliability Criteria – software concern oriented Simplicity S 1 S 2 TCS 2411 Software Engineering S 3 Metrics – quantitative measures 25

Measuring the Factors z The factors are difficult to measure directly z Mc. Call

Measuring the Factors z The factors are difficult to measure directly z Mc. Call defined metrics that are assessed subjectively, on a scale of 0 to 10 z Includes auditability, accuracy, completeness, consistency, modularity, etc. z Factors calculated as follows Fq = c 1 x m 1 + c 2 x m 2 + … + c n x m n where cn = Weighting coefficient (or importance factor) mn = Criteria measured TCS 2411 Software Engineering 26

Alternative Measurements z. Hewlett-Packard developed a set of quality factors - FURPS y. Functionality

Alternative Measurements z. Hewlett-Packard developed a set of quality factors - FURPS y. Functionality - features, capabilities, overall y. Usability - human factors, consistency, doc. y. Reliability - failure, accuracy, MTBF, recovery y. Performance -speed, response time, efficiency y. Supportability - maintainability, testability z. Quality metrics for each step in SE process TCS 2411 Software Engineering 27

Measuring Quality Factors In SE Process Investigations/ Specifications F Design Implementation Testing Support #.

Measuring Quality Factors In SE Process Investigations/ Specifications F Design Implementation Testing Support #. . . %. . . U R P S TCS 2411 Software Engineering 28

Software Quality Assurance z. Quality assurance - essential activity for any business that produces

Software Quality Assurance z. Quality assurance - essential activity for any business that produces products to be used by others z. SQA - a planned and systematic pattern of actions to ensure quality in software z“Quality is Job #1” TCS 2411 Software Engineering 29

Software Review zlike a “filter” for SE process zmust be applied at various points

Software Review zlike a “filter” for SE process zmust be applied at various points during development zall technical work need reviewing zalso checks one’s mistakes zhelps to discover defects early ydesign can introduce 50 - 65% of errors yreview can uncover 75% of them ycost of subsequent steps reduced TCS 2411 Software Engineering 30

ISO 9000 z. Set of standards that can be applied to a range of

ISO 9000 z. Set of standards that can be applied to a range of organisations from manufacturing through to service industries z. ISO 9001 - general standards, applies to design, development, maintaining products z. ISO 9000 -3 - supporting document for software development z. Various countries also have own standards TCS 2411 Software Engineering 31

ISO Certification z. Some countries have bodies which certify that the quality process in

ISO Certification z. Some countries have bodies which certify that the quality process in an organisation conforms to ISO requirements z. In Malaysia, SIRIM performs this function z. Customer looks for certification in an organisation to indicate the quality of products or services TCS 2411 Software Engineering 32

Areas covered by ISO 9001/9000 -3 STANDARD (1/3) A. A. Quality System framework 1.

Areas covered by ISO 9001/9000 -3 STANDARD (1/3) A. A. Quality System framework 1. Management responsibility x quality policy defined, documented, understood, implemented and maintained x responsibilities and authorities for all personnel defined x in-house verification resources defined, trained and funded x a designated manager ensures that the quality program is implemented and maintained 2. Quality system x requires a documented quality system be established x should be an integral process throughout the entire life cycle 3. Internal quality system audits x requires audits be planned and performed x results communicated to management x deficiencies corrected 4. Corrective action x requires causes of non-conforming product be identified x potential causes of non-conformance eliminated x procedures changed as a result TCS 2411 Software Engineering 33

Areas covered by ISO 9001/9000 -3 STANDARD (2/3) B. Quality Life Cycle Activities 1.

Areas covered by ISO 9001/9000 -3 STANDARD (2/3) B. Quality Life Cycle Activities 1. Contract review x contracts are reviewed to determine whether the requirements are adequately defined, agree with the bid, and can be implemented 2. Purchaser’s requirements specification x procedures to identify and collect purchaser’s requirements 3. Development planning x production processes are defined and planned 4. Quality record x quality assurance process is planned 5. Design and implementation x procedures to control and verify the design TCS 2411 Software Engineering 6. 7. 8. 9. x includes planning design activities, identifying inputs and outputs, verifying the design and controlling design changes Testing and validation x requires systems/products to be tested and validated before use x records of testing are kept Acceptance x procedures for acceptance should be established Replication, delivery and installation x procedures for the above should be established and maintained Maintenance x requires that maintenance activities be performed as specified 34

Areas covered by ISO 9001/9000 -3 STANDARD (3/3) C. Quality System Supporting Activities 1.

Areas covered by ISO 9001/9000 -3 STANDARD (3/3) C. Quality System Supporting Activities 1. Configuration management x process of development and maintenance should be documented and controlled 2. Document control x distribution and modification of documents should be controlled 3. Quality records x quality records should be collected, maintained and dispositioned 4. Measurement x where appropriate, adequate statistical techniques should be identified and used to verify the acceptability of process capability and product characteristics TCS 2411 Software Engineering 5. Rules, practices and conventions x are in place and followed 6. Tools and Techniques x are applied appropriately to support the development process 7. Purchasing x purchased products conform to their specified requirements 8. Included software product x should be verified and maintained 9. Training x training needs should be identified and training provided since related tasks may require qualified personnel x training records should be 35 maintained

Quality Management System ISO 9000 Quality Models instantiated as Organisational Quality Manual documents is

Quality Management System ISO 9000 Quality Models instantiated as Organisational Quality Manual documents is used to develop Project 1 Quality Plan Project 2 Quality Plan TCS 2411 Software Engineering Organisational Quality Process instantiated as Project 3 Quality Plan Project Quality Management 36

ISO & SQA z. An organisation that have SQA tasks in place and a

ISO & SQA z. An organisation that have SQA tasks in place and a quality management process would have most of ISO elements z. Ideally quality management should be separate from project management z. This is to ensure quality considerations are not compromised by concerns of budget and schedule TCS 2411 Software Engineering 37

Process Improvement z. Understanding existing processes and changing these processes yto improve product quality

Process Improvement z. Understanding existing processes and changing these processes yto improve product quality yand/or to reduce costs and development time z. Assume that key factor to product quality is the quality of development process zkaizen - process of continuous process improvement TCS 2411 Software Engineering 38

Capability Maturity Model z. Developed by Software Engineering Institute (SEI) of Carnegie-Mellon U z.

Capability Maturity Model z. Developed by Software Engineering Institute (SEI) of Carnegie-Mellon U z. SEI was established to improve the capabilities of US software industry z. Focus on improving the process of software engineering z. Similar to ISO, organisations are assessed and certified to be on Level 1 to Level 5 TCS 2411 Software Engineering 39

Process Maturity Model Level 5 Optimizing Level 4 Managed Level 3 Defined Level 2

Process Maturity Model Level 5 Optimizing Level 4 Managed Level 3 Defined Level 2 Repeatable Level 1 Initial TCS 2411 Software Engineering 40

Level 1 Initial z. No effective management procedures or project plans z. No organisational

Level 1 Initial z. No effective management procedures or project plans z. No organisational mechanisms to ensure consistent use of formal procedures z. Successful software depends more on individual skill rather than organisation TCS 2411 Software Engineering 41

Level 2 Repeatable z. Formal management, quality assurance and configuration control procedures z. Organisation

Level 2 Repeatable z. Formal management, quality assurance and configuration control procedures z. Organisation can successfully repeat projects of the same type z. Lack of formal process model TCS 2411 Software Engineering 42

Level 3 Defined z. Organisation has defined its process and has a basis for

Level 3 Defined z. Organisation has defined its process and has a basis for qualitative process improvement z. Formal procedures are in place to ensure that the defined process is followed in all software projects TCS 2411 Software Engineering 43

Level 4 Managed z. Organisation has a defined process and a formal programme of

Level 4 Managed z. Organisation has a defined process and a formal programme of quantitative data collection z. Process and product metrics are collected and fed into the process improvement activity TCS 2411 Software Engineering 44

Level 5 Optimising z. Organisation is committed to continuous process improvement z. Process improvement

Level 5 Optimising z. Organisation is committed to continuous process improvement z. Process improvement is budgeted and planned and is an integral part of the organisation’s process TCS 2411 Software Engineering 45

Optimizing Key Process Area Process change management Technology change management Defect prevention Managed Software

Optimizing Key Process Area Process change management Technology change management Defect prevention Managed Software quality management Quantitative process management Defined Peer reviews Intergroup coordination Software product engineering Integrated software management Training program Organization process definition Organization process focus Repeatable Software Configuration Management Software Quality Assurance Software Subcontract management Software project tracking and oversight Software project planning Requirements management TCS 2411 Software Engineering Initial 46

Key Process Areas z. Level 1 Initial ynone z. Level 2 Repeatable y. Software

Key Process Areas z. Level 1 Initial ynone z. Level 2 Repeatable y. Software configuration management y. Software quality assurance y. Software subcontract management y. Software project tracking and oversight y. Software project planning y. Requirements management TCS 2411 Software Engineering 47

Key Process Areas (Continued) z. Level 3 Defined y. Peer reviews y. Intergroup coordination

Key Process Areas (Continued) z. Level 3 Defined y. Peer reviews y. Intergroup coordination y. Software product engineering y. Integrated software management y. Training programme y. Organisation process definition y. Organisation process focus TCS 2411 Software Engineering 48

Key Process Areas (Continued) z. Level 4 Managed y. Software quality management y. Quantitative

Key Process Areas (Continued) z. Level 4 Managed y. Software quality management y. Quantitative process management z. Level 5 Optimising y. Process change management y. Technology change management y. Defect prevention TCS 2411 Software Engineering 49

References z“Software Engineering: A Practitioner’s Approach” 5 th Ed. by Roger S. Pressman, Mc-Graw-Hill,

References z“Software Engineering: A Practitioner’s Approach” 5 th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001 z“Software Engineering” by Ian Sommerville, Addison-Wesley, 2001 TCS 2411 Software Engineering 50