Requirements Verification Validation Requirements Engineering CI 2323 Daniel

Requirements Verification & Validation Requirements Engineering (CI 2323) Daniel Siahaan

Content �Introduction to Requirements Engineering �Add-on: Scenario �Feasibility Studies �Requirements Elicitation �Creativity Thinking �Requirements Analysis �Requirements Specification (RS) �Requirements Validation

Tujuan V&V untuk Memastikan (Abran and Moore, 2001) �Dokumen SKPL menjelaskan dengan benar keinginan terhadap kapabilitas dan karakteristik yang memuaskan bagi stakeholder �SKPL diuraikan dengan benar sesuai dengan kebutuhan sistem, aturan bisnis, dan sumber-sumber lain yang berkaitan �Spesifikasi kebutuhan menyeluruh dan berkualitas tinggi �Representasi kebutuhan sudah konsisten dan tidak adanya konflik antar spefikasi satu dengan yang lainnya, �SKPL menyediakan informasi yang cukup, berkualitas dan dapat dipercaya.

Tujuan dari V&V �As an aid to determine that the software requirements are implemented correctly and completely and are traceable �To comprehensively analyze and test the software during the development to determine that the software performs its intended functions correctly, and no unintended function �To provide information about its quality and reliability

Tujuan dari V&V �To ensure that the requirements do not conflict with any standard or requirements of other correlated system. �To analyze, review, and demonstrate, or test all software development output.

Management Task for V&V �Plan and maintain the requirements V&V �Coordinate & interpret performance and quality of the V&V effort �Report discrepancies promptly to the user or development group �Identify early problem trends and focus V&V task on them �Provide a technical evaluation of the software performance and quality attributes at each major software program review �Assess the full impact of proposed software changes

Management Task for V&V �Define the quality and performance objectives �Characterize the types of problems anticipated in the system and define how they would be manifested in the software �Select the software V&V analysis and testing techniques to effectively detect the system and software problem �Select the metrics and techniques applied to V&V results to measure and predict the quality of the software.

Verification and Validation �Verification is �the process of determining whether the model, as a conceptualization or an abstraction, is meaningful and accurate representation of the real system �“Doing the right thing” �Validation �The process of checking the model and the corresponding program(s) to ascertain that they performed as intended. �Is logic of themodel correctly implemented �“Doing the thing right”

Requirements Verification Activities �Conduct a concept documentation evaluation �Begin test planning �Conduct software traceability analysis �Conduct a requirements evaluation �Conduct a interface analysis �Evaluate the reused software �Conduct software interface analysis

Requirements Verification Scope �(S)RS should correctly define all the software requirements �(S)RS should not describe any design or implementation details �(S)RS should not impose additional constraints on the software

Software Verification Principles �Correct �An (S)RS is correct if, and only if, every requirement stated there is one that the system will meet �Unambiguous �An (S)RS is unambiguous if, and only if, every requirements stated there has only one interpretation Should be reviewed by independent party � Write it in a particular requirements specification language (formal methods � Representation tools � �Consistent (Internally) �An (S)RS is internally consistent if, and only if, no subset of individual requirements described in it conflict

Software Verification Principles IEEE 830 -1998 �Lengkap, semua spesifikasi kebutuhan sudah mencakup semua keseluruhan sistem yang dibutuhkan oleh stakeholder. �Diranking berdasarkan kepentingannya, spesifikasi kebutuhan disusun sesuai dengan prioritas pentingnya tiap bagian didalam sistem. �Dapat diverifikasi, hasil akhir dari produk dapat diukur sesuai dengan spesifikasi kebutuhan. �Dapat dimodifikasi, spesifikasi kebutuhan dimungkinkan untuk dilakukan perubahan. �Dapat dilacak, spesifikasi dapat ditelusuri tiap bagiannya tahap demi tahap.

Daftar Pertanyaan untuk Verifikasi Item Ketepatan Verifikasi Ambiguitas Kelengkapan Apakah semua kasus penggunaan dapat direalisasikan melalui sequence diagram? Apakah semua kebutuhan didalam SRS juga terdapat dalam model analisa? Apakah terdapat penggunaan susunan kata seperti “mungkin”, “memungkinkan”, “seharusnya”, “dapat”, “lebih baik”, “beberapa”, “seringkali” atau kata lain yang tidak mempunya ukuran pasti Apakah terdapat susunan kata yang memiliki arti kata yang samar yang memungkinkan multi interpretasi? Apakah sudah menggunakan struktur standar bahasa yang baku? Apakah semua atribut dan method sudah terdapat didalam model analisa yang dispesifikasikan didalam SRS? Apakah semua kondisi dan hubungan keterikatan spesifikasi kebutuhan fungsional terdapat didalam SRS? Apakah semua referensi didefinisikan sepenuhnya? Apakah semua pengertian yang tidak umum didefinisikan?

Daftar Pertanyaan untuk Verifikasi Item Verifikasi Konsistensi Apakah setiap atribut didefinisikan sekali saja? Apakah setiap metode didefinisikan sekali saja? Dapat diverifikasi Apakah semua kuantitas spesifikasi dalam pengertian yang terukur? Sudahkan pengujian kasus telah didefinisikan dalam setiap skenario kasus? Dapat dimodifikasi Apakah struktur SRS sesuai dengan struktur model analisa? Sudahkah semua struktur yang berulang dihilangkan? Dapat dilacak Apakah setiap kebutuhan mempunyai penomoran yang unik dalam artian mengikuti atribut ataupun metode yang digunakan? Apakah setiap kebutuhan dapat ditelusuri pada kasus penggunaannya untuk direalisasikan?

Static and Dynamic Validation �Static: Software inspections. �Concerned with analysis of the static system representation to discover problems �May be supplement by tool-based document and code analysis �Dynamic: Software testing. �Concerned with exercising and observing product behavior �The system is executed with test data and its operational behavior is observed

Requirements checking Types of Checks: � Validity. Does the system provide the functions which best support the customer’s needs? � Consistency. Are there any requirements conflicts or different descriptions of the same function? � Completeness. Are all functions (and constraints) required by the customer included? � Realism. Can the requirements be implemented given available budget and technology? � Verifiability. Can the requirements be checked?

Requirements Verification Techniques � Requirements reviews � Systematic manual analysis of the requirements � Prototyping � Using an executable model of the system to check requirements � Test-case generation Developing tests for requirements to check testability � Tests difficult to implement reveal potential difficulty of implementing requirements � � Automated consistency analysis � Checking the consistency of a structured requirements description (formal notation) using a dedicated CASE tool

Software Verification Techniques �Consistency analysis �Compares the requirements of any existing (used) software with the new software requirements to ensure consistency.

Automated consistency checking

Requirements Reviews �Regular reviews should be held while the requirements definition is being formulated �Both client and contractor staff should be involved in reviews �Reviews may be formal (with completed documents) or informal. Good communications between developers, customers and users can resolve problems at an early stage

Review Checks �Verifiability. Is the requirement realistically testable? �Comprehensibility. Is the requirement properly understood by procurers or end-users? �Traceability. Is the origin of the requirement clearly stated? �Adaptability. Can the requirement be changed without a large impact on other requirements?

Requirements Validation �Concerned with demonstrating that the implementation of the requirements as what the client espects �Requirements error costs are high so validation is very important �Fixing a requirements error after delivery may cost up to 100 times the cost of fixing an implementation error (it goes through specs, design and implementation)

Validation can only reveal the presence of error not their absence

Software Validation Techniques �Algorithm analysis �Examines the logic and accuracy of the software requirements by tranlating algorithms into some language structured format �Back-to-back testing �Detects test failures by comparing the output of two or more programs implemented to the same specification �Boundary value analysis �Detects and removes errors occurring at parameter limits or boundaries.

Software Validation Techniques �Database analysis �Analysis and validate data model. �Data use analysis �Detects uninitialised variables, variables written twice without an intervening assignment, variables which are declared but never used, etc �Decission (truth) table �Analyze and validate the logical of decission-making behaviour of the system.

Software Validation Techniques �Control flow analysis �Checks for loops with multiple exit or entry points, finds unreachable code, etc. �Coverage analysis �Measures how much of the structure of a unit or system has been exercised by a given set of tests � (B. Marick. The Craft of Software Testing, Subsystem testing Including Object- Based and Object-Oriented Testing. Prentice-Hall, 1985. )

Software Validation Techniques �Error seeding �The process of intentionally adding known faults to those already in a computer program for the purpose of monitoring the rate of detection and removal, and estimating the number of faults remaining in the program. �Mutation analysis �Modifying programs' source code or byte code in small ways, so that any tests which pass after code has been mutated are considered defective. The purpose is develop effective tests or locate weaknesses in the test data used for the program or in sections of the code that are seldom or never accessed during execution.

Software Validation Techniques �Interface analysis �Checks the consistency of routine and procedure declarations and their use �Interface Testing �Checks whether necessary input/output is presence

Software Validation Techniques �Event tree analysis �Provides an inductive approach to reliability assessment given a graphical representation of the logic model that identifies and quantifies the possible outcomes following an initiating event. �Finite State Machine �a model of behavior composed of a finite number of states and transitions between those states

Software Validation Techniques �Information flow analysis. �Identifies the dependencies of output variables. Does not detect anomalies itself but highlights information for code inspection or review �Path analysis. �Identifies paths through the program and sets out the statements executed in that path. Again, potentially useful in the review process

Kakas Bantu: SCR (Heitmeyer et al, 2007) �Software Cost Reduction

Kakas Bantu: Qu. ARS (Lami et al, 2005)

Kakas Bantu: Req. SAC (Hussain et al, 2007)