Better Translating Customer Expectations into Design Requirements Lawrence

Better Translating Customer Expectations into Design Requirements Lawrence Chung Dept. of Computer Science Erik Jonsson School of Engineering and Computer Science The University of Texas at Dallas For Huawei, Summer 2007

Customer Expectations? Ø With cellular phones: “… enhancements enable the best possible operation of your mobile … in various conditions. … The earpiece fits in either ear allowing for convenient and reliable access to all basic call controls. . To maximize call security, the headset also supports … the wireless connection for compatible … models. ” Ø With home networking: “… is the total home networking solution … linking variety of digital home appliances as one. It enables you to enjoy convenient, pleasant, comfortable, and reliable living environment at any time and any place.

What does “reliability” mean? • Customer Expectations : according to J. Musa the ability of the system to behave consistently in a user-acceptable manner when operating within the environment for which the system was intended. A Big Gap! • Design Requirements: theory and practice of hardware reliability are well established; some try to adopt them for software - one popular metric for hardware reliability is mean-time-to-failure (MTTF) MTTR MTTF MTBF • Availability = [MTTF/(MTTF + MTTR)] x 100% Sometimes reliability requirements take the form: "The software shall have no more than X bugs/1 K LOC" Measure bugs at delivery time, based on a Monte Carlo statistical analysis of random events.

Outline Issues Major Issues – The Standish “Chaos” Report Process Requirements Engineering Process FR Functional Requirements What should be done (e. g. , use cell to control garage door) NFR Non-Functional Requirements How should it be done (e. g. , use cell to control garage door, “fast” and “in a secure manner”)

Issues Do Requirements Matter? If so, how do we make them better? The Standish Group Report, ‘ 01 – The Chaos Report (www. standishgroup. com), yearly since 1994 Project Success Factors The CHAOS Ten 1. Executive Management Support 28% completed on time and on budget 78, 000 projects canceled before completion 65, 000 projects 23% 2. User Involvement 3. Experienced Project Manager overran original estimates 4. Clear Business Objectives -Time overrun averaged 63% - Cost overrun averaged 45% 5. Minimized Scope 6. Standard Software Infrastructure 137, 000 projects 49% 7. Firm Basic Requirements 8. Formal Methodology 9. Reliable Estimates 10. Other

Issues What Factors Contribute to Project Failure? The CHAOS Ten Standish Group, ‘ 01 (www. standishgroup. com) “The definition of insanity is doing the same thing over and over again and expecting a different result. ” [Albert Einstein]

Issues Better Translating Customer Expectations into Design Requirements More Complete … and … More [IEEE 830 -1993, § 4. 3. 2, 1994] • Correct – Customer Expectations • Complete – • • ØFunctional • ØNon-Functional • ØQuality (incl. Qo. S) ØCost ØTime-to-Market Ø… • Design Requirements Does not conflict with other requirements. Unambiguous – Is subject to one and only one interpretation. Verifiable – Can be tested cost effectively. Ranked for importance and stability – Can be sorted based on customer importance and stability of the requirement itself. Modifiable – Changes do not affect the structure and style of the set. Traceable – • Describes all significant requirements of concern to the user. Consistent – • Is a true statement of something the system must do. The origin of each requirement can be found. Understandable – Comprehended by users and developers.

Process Requirements Engineering Process v 3 fundamental activities: understand, (formally) describe, attain an agreement on, the problem Many variations and extensions User reqs Elicitation knowledge User Specification For more knowledge User feedback Req. models Val. result Validation Domain knowledge Problem Domain (domain experts, laws, standards, policies, documents, etc. ) Domain knowledge • • Elicitation: determine what’s really needed, whom to talk to Specification: produce a (formal) RS model: translate "vague" into "concrete", etc. ; make various decisions on what & how • Validation: assure that the RS model satisfies the users’ needs

Process From Customer Expectations into Design Requirements: A Roadmap An interleaving and continuous loop: elicit, model, translate, specify, validate Customer/ Business Problem Customer/Business problem & expectations elicited, negotiated, defined, validated in the context of the customer/business goals traceability! §Identify stakeholders for problem. §Root cause analysis. Establish common vocabulary => Domain Model §Identify constraints on the system/project §Define the solution system boundary Alternative solutions explored, negotiated, defined, validated Best solution chosen to meet the goals Customers do NOT really know what they want: Help them find out what they may expect Customers do NOT really know about the solution: Help them find out what choices they have

FR Understanding and Modeling Functional Requirements Customer/Business Modeling Using UML, the de facto industry standard Object-oriented, visual language, facilitates communication Describe the Problem in the Vision Document Stakeholder Requests • Communicates information between all parties – customer, management, marketing, and the project team. • Provides initial customer feedback. • Fosters general understanding of the product. • Establishes scope and priority of high-level stakeholder requests and features. • A system-level document that describes the “what” and “why” of the product. • Non-functional requirements are not treated as first-class citizens. Vision Document Use-Case Model Design Specifications Supplementary Specification User Documentation Specifications

FR Understanding and Modeling Functional Requirements Customer/Business Modeling Using UML, the de facto industry standard A consortium of communicating agents Workflow Model Scenarios Adapted from [Copyright © 1987 - 2001 Rational Software Corporation] Use-Case Model Dynamic Model Object Model

NFR Understanding and Modeling Non-Functional Requirements Ø With cellular phones: “… enhancements enable the best possible operation of your mobile … in various conditions. … The earpiece fits in either ear allowing for convenient and reliable access to all basic call controls. . To maximize call security, the headset also supports … the wireless connection for compatible … models. ” Ø With home networking: “… is the total home networking solution … linking variety of digital home appliances as one. It enables you to enjoy convenient, pleasant, comfortable, and reliable living environment at any time and any place.

NFR What are Non-Functional Requirements? F: I O add: 2, 5 7 • -ilities: understandability, usability, modifiability, inter-operability, reliability, portability, dependability, flexibility, availability, maintainability, scalability, (re)configurability, customizability, adaptability, stability, variability, volatility, traceability, verifiability, predictability, compatibility, survivability, accessibility, … • -ities: security, simplicity, clarity, ubiquity, integrity, safety, modularity, nomadicity, … • -ness: user-friendliness, robustness, timeliness, responsiveness, correctness, completeness, conciseness, cohesiveness, … • …and there are many more: convenience, comfort, performance, efficiency, accuracy, precision, slim, esthetics, consistency, coherence, fault tolerance, self-healing capability, autonomy, cost, development time, time-to-market, long-lasting battery, low coupling, … ambiguous, subjective, conflicting, global

NFR NFRs: subjective in both definitions & solutions Classification 1 [Roman, IEEE Computer 1985] 6 classes + 2 -3 levels Classification 2 - Process, Product and External considerations [Sommerville 1992] 3 classes + 2 levels

NFR NFRs: subjective in both definitions & solutions Classification 3 3 classes + 3 levels Note correlations

NFR NFRs: subjective in both definitions & solutions Classification 4 5 classes + 2 levels Dimensions of Quality –Components of FURP+ [Grady 1992] F unctionality Feature set capabilities, security, generality U sability Human factors aesthetics, consistency, documentation Reliability Frequency/severity of failure, recoverability, predictability, accuracy, MTBF Performance Supportability Speed efficiency, resource usage, throughput, response time Testability Adaptability Compatibility Serviceability Localizability Extensibility Maintainability Configurability Installability Robustness

NFR NFRs: subjective in both definitions & solutions Classification 5 3 classes + 3 levels Software Quality Tree [Boehm 1976] Note correlations

NFR Relationship Between Design Goals Adapted from [Brugge] One possibility Client (Customer, Sponsor) Low cost Increased Productivity Backward-Compatibility Traceability of requirements Rapid development Flexibility Runtime Efficiency Functionality End User-friendliness Ease of Use Ease of learning Fault tolerant Robustness Reliability Portability Good Documentation Minimum # of errors Modifiability, Readability Reusability, Adaptability Well-defined interfaces Developer/Maintainer User

NFR Goal-Oriented Requirements Engineering: The NFR Framework, Tropos, KAOS, … Softgoal Interdependency Graph (SIG) Object Model FR From NFR Softgoals to Op Softgoals (here, Use Cases) ambiguous, subjective, conflicting, global § Treat NFRs as softgoals § Refine them traceably § Achieve them functionally § Detect and resolve conflicts § Prioritize throughout! E. g: A small portion of a hospital model for requirements analysis J. Mylopoulos, L. Chung, E. Yu, "From object-oriented to goal-oriented requirements analysis ", CACM, pp 31 -37. ACM Press

NFR Ø • • • • • • NFRs – Do’s & Don’ts Dos Relate to FRs Clarify scope/topic Identify agents, whenever useful Discover relationships between definitions of NFRs Discover relationships between solutions to NFRs Refine definitions as many times as needed Refine solutions as many times as needed Prioritize Discover conflicts Safeguard against conflicts Discover synergies Discover operationalizations as reasons for conflicts/synergies Determine strengths of contributions Justify strengths of contributions Explore alternatives Discover solutions from requirements Discover requirements from solutions Consider use of multiple solutions Consider scenarios If necessary, quantify Evaluate, …subjectively, …objectively Establish traceability Ø Don’ts • • • • Absolute security, absolute reliabilty, absolute safety, …. One definition fits all One solution solves all the problems The contribution is such and such, since I say so Refine the definition only once They are falling down from the sky Dissociate from FRs May be more important than FRs, but should consume less resources You name it; our system does it No quantification, no existence Everybody needs the same Be only pessimistic Asking why “+” reveals ignorance Beg the question Evaluate & only evaluate Brainwash nothing but objectivity

FR NFR Theoretical Foundations FR The 4 -variable model: q M REQ complete if SOF m M (REQ(m) = OUT(SOF(IN(m)))) C IN OUT I SOF O FR The reference model (WRSPM): complete if q P, C |= S C – Computer D – Domain Properties (World, Enterprise, Business, Domain theory) S - Specification P - Program R - Requirements FRq NFR The goal model: If R denotes the set of requirements, As the set of assumptions (expectations), S the set of software specifications, Ac the set of accuracy goals, and G the set of goals complete if

Issues Process FR NFR Summary ØBetter Translating Customer Expectations into Design Requirements needs ØBetter Methodologies & Notations Towards More Complete … and … More Customer Expectations ØFunctional (Object-Oriented) ØNon-Functional (Goal-Oriented) Design Requirements Correct, Consistent, Unambiguous, Verifiable, Ranked for importance and stability, Modifiable, Traceable, Understandable

Issues Process FR NFR On-going Work ØComponent Reuse and Interoperability ØRequirements & Architecture Patterns ØVerification Using Model Checking ØSecurity and Privacy Requirements Engineering

NFR: Appendix Dependability • Dimensions of Dependability – – Availability: The ability of the system to deliver services when requested Reliability: The ability of the system to deliver services as specified Safety: The ability of the system to operate without catastrophic failure Security: The ability of the system to protect itself against accidental or deliberate intrusion • Cost of development - Geometric rise in cost from low dependability to highest • Effects of low dependability – – Often unused Failure recovery costs may be high Difficult to retrofit dependability Loss of information • Repeatable improvement process helps • Critical Systems – Safety critical – Mission critical – Business critical • Dependability a key aspect – A system failure causes • • • Significant economic loss Physical damage Threat to or loss of human life

Process From Customer Expectations into Design Requirements: A Roadmap An interleaving and continuous loop: elicit, translate, model, specify, validate §Identify stakeholders for problem. §Root cause analysis. Customers do NOT really know what they want Help them find out what they may expect §Gain agreement on the problem & expectation def. wrt. root causes §Consider alternatives & choose the best solution(s) to meet the goals. §Understand the problem & expectations in the context of the customer/business goals. §Identify constraints on the system/project Customers do NOT really know what the solution might be like Help them find out what choices they have §Define the solution system boundary Elicit Requirements

Process From Customer Expectations into Design Requirements: A Roadmap An interleaving and continuous loop: elicit, translate, model, specify, validate Customer/ Business Problem §Identify stakeholders for problem. §Root cause analysis. Customer/Business problem & expectations defined §Understand the problem & expectations in the context of the customer/business goals. §Identify constraints on the system/project §Gain agreement on the problem & expectations def. wrt. root causes §Consider alternatives & choose the best solution(s) to meet the goals. Problem & expectations validated/adjusted Best solution identified Elicit Requirements §Define the solution system boundary q Context: Establish common vocabulary => Glossary with a Domain Model