Requirements Analysis Tool A Tool for Automatically Analyzing

Requirements Analysis Tool: A Tool for Automatically Analyzing Software Requirements Documents Kunal Verma, Alex Kass Copyright © 2008 Accenture All Rights Reserved. Accenture, its logo, and High Performance Delivered are trademarks of Accenture.

Outline • Introduction • Lexical and Syntactic Analysis • Semantic Analysis – Conflict detection – Missing requirements – System integration diagrams • Conclusions Copyright © 2008 Accenture All Rights Reserved. 2

INTRODUCTION Copyright © 2008 Accenture All Rights Reserved. 3

Poor Requirements are a Major Factor in Software Quality and Cost • Requirements problems drive up costs and stretch timelines – 82% of rework effort is caused by requirements defects 1 – Cost overruns begin to increase dramatically when less than 8% of project cost is devoted to the requirements engineering process 2 – Increased use of globally distributed model is likely to intensify the problem 1 Source: “Borland: Best Practices for Requirements Development & Management” “Borland: Ivy Hooks: 28 NASA Projects” 2 Source: Copyright © 2008 Accenture All Rights Reserved. 4

Requirements Documents and People Stakeholders Responsibilities: • Participate in elicitation sessions to outline requirements • Sign-off document requirements Profile: • Deep business • Low to medium technical Requirements Team/ Business Analysts Requirements Document Responsibilities: • Document requirements • Organize, analyze and prioritize requirements • Obtain sign-off from stakeholders Skills: • Deep business • Low to medium technical Copyright © 2008 Accenture All Rights Reserved. Design Team Responsibilities: • Understand requirements to create design documents Skills: • Low to medium business • Medium to high technical Responsibilities: • Understand requirements and design to create software Skills: • High technical Development Team

Poor Requirements can Cause Problems 3 kinds of requirements-documentation best practices have been developed to help avoid these problems • Lexical Avoid vague words and phrases • Do not use terms such as “easy to use”, “flexible”, etc. • Use consistent terminology • Do not use Order Processing System and Order Entry System to refer to the same system • Help stakeholders see certain semantic relationships • Increasing security affects Semantic time • Identify potential conflicts • Platform x doesn’t support WS-Policy Copyright © 2008 Accenture All Rights Reserved. Best Practices • Use simple, standardized sentence structure Syntactic • Write “The System (agent) shall generate (action verb) a report” instead of “report will be generated”. 6

Dilemma: Two Options for Requirements Analysis 1. Write requirements in formal notation – – – Ease of analysis Unrealistic due to people involved Formal notation hasn’t been shown to scale 2. Allow analysts to write in natural language – – Ease of writing Analysis is intractable This paper leverages best practices to simplify analysis: – Set of controlled syntaxes – User-defined glossaries – Structured content extracted analyzed Copyright © 2008 Accenture All Rights Reserved. 7

LEXICAL AND SYNTACTIC ANALYSIS Copyright © 2008 Accenture All Rights Reserved. 8

Examples of Analysis Performed by RAT Copyright © 2008 Accenture All Rights Reserved. 9

Controlled Syntaxes Supported by RAT Requirement Type Solution Requirement Enablement Requirement Action Constraint Format and Examples Agent. Phrase <“shall” | “must”> Action. Phrase Agent. Phrase <“shall” | “must”> “be able to” actionphrase Agent. Phrase <”shall” | “must”> <”allow” | “permit”> agentphrase actionphrase Policy ER-1: The user must be able to display the PDF rendition of associated documents. ER-3: MMMS shall allow users to add new memberships. Agentphrase <“shall only” | “may only”” | “shall not”> actionphrase <”when” | “if”> condition. “Only” agentphrase “may” [“be”] Action. Phrase Attribute Constraint Definition SAR 1: The order processing system shall wait for an acknowledgement from the payment processing system. SAR 2: Develop Bills System shall produce invoices with Rolling Rates. SCR 1: The account management system shall only close an account if the current balance is zero BC 2: Only payroll employees may access the payroll database. Entity. Phrase “must” <”always“| “never” | “not”> <”be” | “have”> entityphrase AC 1: Customer standing must always be one of the following: 1) Gold 2) Sliver 3) Bronze. AC 2: The Order Report must always have the following fields: 1) Order Number 2) Order Quantity 3)Delivery Address Entity. Phrase <“is” | “will be”> <“defined as” | “classified as”> entity-phrase Def 1: Total sale value is defined as total item value plus sales tax Entityphrase “is” [“not”] actionphrase P 1: Salestax is computed on in-state shipments. P 2: Salestax is not computed on interstate shipments Copyright © 2008 Accenture All Rights Reserved. 10

Sample Glossaries Agent Glossary Agent Name Description Class Super-Class Web Server The Web Server for the project. System Agent Payroll Employee The payroll dept employee User Agent SAP System The SAP system for the project System Agent Order Parts Process The ordering parts process Process Agent Problem Phrase Glossary Problem Phrase Explanation Suggestion easy easily easy to user friendly is often ambiguous; consider replacing with a specific description of the expected user profile, and expected time to competence or completion. A user with <specify background> will be able to <specify function> with <specify effort> The system will reduce the effort required to <specify function> by x% The system will require no more than <specify duration> time for a <specify user profile> to learn to use. quickly Fast Rapid is often ambiguous; consider replacing with a specific response time. Example: 'within 5 seconds' Should Could May is often ambiguous, or inappropriate. Some readers will interpret this as optional or advisory, others as required. Use 'shall' for requirements, 'will' for assumptions. Is ambiguous. Which kind of encryption? within <x> milliseconds within <x> minutes within <x> hours within <x> days within <x> weeks will shall Strong encryption Copyright © 2008 Accenture All Rights Reserved. The system provide encryption using SSH The system will provide encryption using RSA 11

Syntactic Analysis Approach Requirements Document Glossaries SA 1: The SAP System shall send vendor data to the order processing system. Tokenization Lexical Analysis Classification Syntactic Analysis Copyright © 2008 Accenture All Rights Reserved. SA 1: The SAP System shall send vendor data to the order processing system. 12

SEMANTIC ANALYSIS Copyright © 2008 Accenture All Rights Reserved. 13

Semantic Analysis • Use the extracted structured content from the parser to create a semantic graph in RDF • Using Jena and SPARQL for: – Requirements Dependency Analysis • Finding relationships such as conflicts between requirements – Finding missing requirements – Generating system interaction diagrams Copyright © 2008 Accenture All Rights Reserved. 14

From Requirements to RDF Graph Core requirements concepts Agent taxonomy from agent glossary Requirement relationships from requirements relationship glossary Instances from requirements documents The Web Server shall encrypt all its responses using SSH. The Web Server shall have a response time of 5 milliseconds or less. Copyright © 2008 Accenture All Rights Reserved. 15

Dependencies and Conflicts Analysis • Problem: – Requirements documents can contain many hard-to-detect dependencies conflicts between existing requirements. • Solution: – SMEs develop of repository of common relationships between various kinds of requirements • E. g. LDAP based authentication increases response time – Tools applies these relationships to automatically detect potential conflicts in a particular requirements set. • Challenges: – Must make it easy for SME’s to build the repository by entering information about relationships in order to scale this approach – Create a framework to manage distributed rule creating(future work) Copyright © 2008 Accenture All Rights Reserved. 16

Can you Find the Dependencies in these Requirements? A 1: Workflow designer shall use platform X for developing workflows. A 2: Workflow designer shall support SAML for enforcing security. A 3: SAP System shall use SSH for security. A 4: SAP System shall respond in 5 milliseconds. q. Sample SOA relationships (from Accenture’s research): q. SAML has unresolved errors with platform X. q. Hence A 1 conflicts with A 2 q Dependencies from non-functional requirement glossary: q Security requirements increase response time q Hence, A 3 and A 4 may be in conflict Copyright © 2008 Accenture All Rights Reserved. 17

Requirements Relationship Glossary • Requirements Relationship Glossary used for entering concept hierarchies –Binary relationships and keywords associated with each concept Requirement Class Requirement super. Keywords Non-Functional class Security Authentication Non. Functional Security Encryption Security Time Query. Time Response. Time Authentication Non. Functional Security Time Requirement Relationship Affects: Time Password, token, authentication, kerberos Encryption, encrypt, SSH, Increases: Response. Time RSA, DSA, PGP, keys affects Time Query time, querytime Response time, responsetime, respond Encryption Copyright © 2008 Accenture All Rights Reserved. Response Time Query Time 18

Dependency Analysis: An Example • Consider the following two requirements – The Web Server shall encrypt all its responses using SSH. – The Web Server shall have a response time of 5 milliseconds or less. • SPARQL query select ? req 1, ? req 2 where { ? req 1 has. Requirement. Type ? type 1. ? req 2 has. Requirement. Type ? type 2. Affects domain ? type 1. Affects range ? type 2. ? req 2 has. Agent ? agent 2. ? req 1 has. Agent ? agent 1 filter( ? agent 1 = ? agent 2)}) Copyright © 2008 Accenture All Rights Reserved. 19

Systems Based Analysis • Which systems are interacting with other systems? • Which Systems that are missing certain kinds of nonfunctional requirements? • Are there any interacting systems that do not have compatible security profiles? Copyright © 2008 Accenture All Rights Reserved. 20

Systems Based Analysis Example: Finding Interacting Requirements When the analysts writes these requirements… 1: The Web Server shall send the vendor data to the SAP System. 2: The Web Server shall store order requests in the vendor database. 3: The Web Server shall send the order data to Ariba. 4: The SAP System shall retrieve order data from Ariba. …the system generates this diagram: SAP System Vendor Database Web Server Ariba Copyright © 2008 Accenture All Rights Reserved. 21

Systems Based Analysis Example: Finding Missing Non-functional Requirements MISSING FUNCTIONAL REQUIREMENTS REPORT System SAP System Web Server Ariba Vendor database Encryption Authentication Performance Availability SSH Missing 2 ms 0. 99 RSA LDAP 0. 1 ms Missing RSA LDAP 2 ms 0. 90 RSA LDAP 4 ms 0. 999 Copyright © 2008 Accenture All Rights Reserved. 22

Enterprise Knowledge can be Modeled to Drive Semantic Analysis • Consider some of the results from project Tarpon: – – – Platform A unable to apply WS-Policy to WSDL service Platform B and Platform C do not support WS-Policy Platform D does not support WS-Addressing Platform A and Platform B do not support WS-Metadata. Exchange Incompatibility within WS-Reliability and WS-Reliable. Messaging • This information can be codified using RDF Graphs and used for finding conflicting requirements based on the results using SPARQL queries: – Messaging System shall use Platform B. – Messaging System shall use WS-Policy for providing security policy. Copyright © 2008 Accenture All Rights Reserved. 23

CONCLUSIONS AND FUTURE WORK Copyright © 2008 Accenture All Rights Reserved. 24

Current State of RAT • RAT (with syntactic analysis) has been used at 8 projects – Semantic version out next year • Initial results are promising – No issues found in creating glossaries – Reporting up to 50% time savings in requirements reviews – Over 10, 000 requirements have been run through RAT Copyright © 2008 Accenture All Rights Reserved. 25

Conclusions and Future Work • Poorly documented requirements is a real problem • RAT allows users to write requirements in natural language – Leverages controlled syntaxes and user glossaries • RDF and SPARQL provide framework for codifying enterprise knowledge and rules – Trade-offs between expressive queries and ease-of-use • Future work – Other visualizations – Impact analysis Copyright © 2008 Accenture All Rights Reserved. 26