Using Interactive Genetic Algorithm for Requirements Prioritization Outline

Using Interactive Genetic Algorithm for Requirements Prioritization Outline Using Interactive Genetic Algorithm for Requirements Prioritization Master’s Thesis Presentation Problem Related Works Genetic Algo Francis Palma Our Approach Case Study Results Department of Information Engineering and Computer Science Universitá degli Studi di Trento francis. palma@studenti. unitn. it Discussions Conclusions December 3, 2010

Presentation Outline Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo q Problem description q Related works q Genetic Algorithm q Our approach q Case study q Results q Conclusions Our Approach Case Study Results Discussions Conclusions

Problem Description Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Problem: [Prioritization of Requirements] To find the best ordering of requirements in each successive release to ensure quality & value of the delivered system, trade-off constraints & end-user satisfaction.

Problem Description (cont) Using Interactive Genetic Algorithm for Requirements Prioritization 1. 2. 3. Acquisition and coding of set of Requirements and Domain Knowledge Apply A Prioritization Technique Output of the ranking Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Why Ordered List? Order of implementation satisfy the developers’ constraints & delivers maximum value to the user. Find it!

Domain Knowledge: Prio & Dep Using Interactive Genetic Algorithm for Requirements Prioritization Domain knowledge includes two precedence graphs • • Prio Dep How to build them? Outline Prio Problem Related Works Genetic Algo Prio: For 1 st release, initial priorities Our Approach Case Study Results Discussions Conclusions Dep: For 1 st release, dependencies Dep

Classification: State-of-the-Art approaches Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Prioritization Approaches Classification using user knowledge either performing pair-wise comparison or not using domain knowledge Genetic Algo Our Approach using both Case Study Results Discussions Conclusions ü User Knowledge refers to the awareness of the requirements attributes and the overall system functionalities to be developed

State-of-the-Art approaches (cont) Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Pairwise Comparison based approaches Analytic Hierarchy Process (AHP): involves comparing all unique pairs of requirements to determine which of the two is of higher priority, and to what extent Bubble Sort: compares two requirements & swap them if they are in the wrong order Cost-Value Approach: each individual requirement is determined on (i) the value to the users (ii) the cost of implementing the requirements. It uses the AHP technique Case Based Ranking (CBRank): exploits a machine learning algorithm to guide the elicitation of user preferences during the prioritization process

State-of-the-Art approaches (cont) Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Non-Pairwise comparison based approaches Numerical Assignment: grouping requirements into different priority groups Mo. Sco. W: groups all requirements into four priority groups MUST have, SHOULD have, COULD have, and WON’T have. Simple Ranking: requirements are simply ranked from integer 1 to N Binary Search Tree: each node represents a requirement, requirements placed in the left subtree of a node are of lower priority and those placed in the right subtree of a node are of higher priority than the node priority. $100 Method: each stakeholder is asked to assume having $100 to distribute over the requirements in a ratio scale Combining Techniques based approaches Planning Game: combination of two prioritization techniques i. e. Numerical Assignment & Simple Ranking Domain Knowledge based approaches Priority Groups: dividing requirements into separate groups. then groups are ranked by using AHP Genetic Algorithm: optimization is an application of GA & used in the problem of requirements prioritization too; uses domain knowledge

State-of-the-Art approaches (cont) Using Interactive Genetic Algorithm for Requirements Prioritization Summary Approach Cons AHP Scalability Outline Bubble Sort Scalability, time Problem CBRank unable to accept constraints i. e. Dep Related Works Cost-Value Approach Time consuming Genetic Algo Our Approach Case Study Results BST Sensitivity; a single error may build wrong tree GA Can’t resolve contradictory; $100 Method Longer; less confidence, biased Mo. Sco. W Ambiguous final ordering Simple Ranking unable handling complex scenarios Discussions Conclusions Scalability is a very common problem of all the approaches!

Genetic Algorithm Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions 1. 2. 3. Acquisition and coding of set of Requirements and Domain Knowledge Apply Genetic Algorithm (no user knowledge) Output of the ranking (the most promising individual)

Genetic Algorithm Pseudo Code Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions The Canonical GA (a simple pseudo code is presented here): 1. choose initial population 2. evaluate each individual’s fitness REPEAT: 3. select best-ranking individuals to reproduce 4. apply crossover operator 5. apply mutation operator 6. evaluate each individual’s fitness until terminating condition (e. g. until at least one individual has the desired fitness or enough generations have passed)

What is Population & Individuals? Using Interactive Genetic Algorithm for Requirements Prioritization Indv. ID Individual Pr 1 < R 1, R 3, R 2, R 4, R 5, R 6, R 7, R 8, R 9 > Pr 2 < R 2, R 3, R 4, R 1, R 5, R 8, R 6, R 7, R 9 > Pr 3 < R 5, R 2, R 1, R 3, R 7, R 8, R 6, R 9, R 4 > Pr 4 < R 4, R 5, R 6, R 3, R 2, R 1, R 8, R 9, R 7 > Outline Pr 5 < R 7, R 8, R 6, R 5, R 2, R 3, R 4, R 9, R 1 > Problem Pr 6 < R 5, R 6, R 7, R 8, R 9, R 1, R 2, R 3, R 4 > Pr 7 < R 9, R 8, R 7, R 6, R 5, R 4, R 3, R 2, R 1 > Pr 8 < R 8, R 9, R 6, R 7, R 4, R 5, R 2, R 3, R 1 > Genetic Algo Pr 9 < R 1, R 3, R 5, R 7, R 9, R 2, R 4, R 6, R 8 > Our Approach Pr 10 < R 1, R 4, R 2, R 3, R 5, R 6, R 9, R 8, R 7 > Related Works Disagree Population! A set of solution candidates Case Study Results Pr 1 R 3 Discussions Conclusions R 2 R 4 R 5 R 6 R 7 R 8 R 9 An individual, also a solution candidate A requirement as a gene

Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions How we define Individual’s Fitness?

Domain Knowledge Encoding Using Interactive Genetic Algorithm for Requirements Prioritization Transform the domain knowledge into graphs Req Priorities Dependencies R 1 10 R 2, R 3, R 7 R 2 20 R 3 10 Outline R 4 20 Problem R 5 30 R 6 40 R 7 30 R 8 20 R 9 50 Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions R 4, R 8 R 3 R 8, R 5

Disagreement Calculation Indv. ID Individual Pr 1 < R 1, R 3, R 2, R 4, R 5, R 6, R 7, R 8, R 9 > Pr 2 < R 2, R 3, R 4, R 1, R 5, R 8, R 6, R 7, R 9 > Pr 3 < R 5, R 2, R 1, R 3, R 7, R 8, R 6, R 9, R 4 > Pr 4 < R 4, R 5, R 6, R 3, R 2, R 1, R 8, R 9, R 7 > Pr 5 < R 7, R 8, R 6, R 5, R 2, R 3, R 4, R 9, R 1 > Pr 6 < R 5, R 6, R 7, R 8, R 9, R 1, R 2, R 3, R 4 > Problem Pr 7 < R 9, R 8, R 7, R 6, R 5, R 4, R 3, R 2, R 1 > Related Works Pr 8 < R 8, R 9, R 6, R 7, R 4, R 5, R 2, R 3, R 1 > Pr 9 < R 1, R 3, R 5, R 7, R 9, R 2, R 4, R 6, R 8 > Pr 10 < R 1, R 4, R 2, R 3, R 5, R 6, R 9, R 8, R 7 > Using Interactive Genetic Algorithm for Requirements Prioritization Outline Genetic Algo Disagree Prio Our Approach Case Study Results Discussions Conclusions For Pr 1 R 3 R 2 R 4 R 5 R 6 R 7 R 8 R 9 Conflicts = {(R 5, R 8), (R 6, R 7), (R 6, R 8), (R 7, R 8)}

Disagreement Calculation (cont) Indv. ID Individuals Pr 1 < R 1, R 3, R 2, R 4, R 5, R 6, R 7, R 8, R 9 > Pr 2 < R 2, R 3, R 4, R 1, R 5, R 8, R 6, R 7, R 9 > Pr 3 < R 5, R 2, R 1, R 3, R 7, R 8, R 6, R 9, R 4 > Pr 4 < R 4, R 5, R 6, R 3, R 2, R 1, R 8, R 9, R 7 > Pr 5 < R 7, R 8, R 6, R 5, R 2, R 3, R 4, R 9, R 1 > Pr 6 < R 5, R 6, R 7, R 8, R 9, R 1, R 2, R 3, R 4 > Problem Pr 7 < R 9, R 8, R 7, R 6, R 5, R 4, R 3, R 2, R 1 > Related Works Pr 8 < R 8, R 9, R 6, R 7, R 4, R 5, R 2, R 3, R 1 > Pr 9 < R 1, R 3, R 5, R 7, R 9, R 2, R 4, R 6, R 8 > Pr 10 < R 1, R 4, R 2, R 3, R 5, R 6, R 9, R 8, R 7 > Using Interactive Genetic Algorithm for Requirements Prioritization Outline Genetic Algo Disagree Dep Our Approach Case Study Results Discussions Conclusions For Pr 1 R 3 R 2 R 4 R 5 R 6 R 7 R 8 R 9 Conflicts = {(R 1, R 3), (R 1, R 7), (R 1, R 2), (R 5, R 8)}

Disagreement Calculation (cont) Using Interactive Genetic Algorithm for Requirements Prioritization § Disagreement: Between a pair of ordering i. e. Pr 1 and Prio, disagreement is the count of pairs that are inverted in two orderings. Lower disagreement defines higher fitness. Formally define: Outline Indv. ID Individuals Disagree Problem Pr 1 < R 1, R 3, R 2, R 4, R 5, R 6, R 7, R 8, R 9 > 8 Pr 2 < R 2, R 3, R 4, R 1, R 5, R 8, R 6, R 7, R 9 > 8 Pr 3 < R 5, R 2, R 1, R 3, R 7, R 8, R 6, R 9, R 4 > 16 Genetic Algo Pr 4 < R 4, R 5, R 6, R 3, R 2, R 1, R 8, R 9, R 7 > 15 Our Approach Pr 5 < R 7, R 8, R 6, R 5, R 2, R 3, R 4, R 9, R 1 > 23 Pr 6 < R 5, R 6, R 7, R 8, R 9, R 1, R 2, R 3, R 4 > 29 Pr 7 < R 9, R 8, R 7, R 6, R 5, R 4, R 3, R 2, R 1 > 30 Results Pr 8 < R 8, R 9, R 6, R 7, R 4, R 5, R 2, R 3, R 1 > 29 Discussions Pr 9 < R 1, R 3, R 5, R 7, R 9, R 2, R 4, R 6, R 8 > 17 Pr 10 < R 1, R 4, R 2, R 3, R 5, R 6, R 9, R 8, R 7 > 13 Related Works Case Study Conclusions … and so on … Total Conflicts = {(R 5, R 8), Prio (R 6, R 7), (R 6, R 8), (R 7, R 8), (R 1, R 3), Dep (R 1, R 7), (R 1, R 2), (R 5, R 8)}

GA Crossover Operator Using Interactive Genetic Algorithm for Requirements Prioritization We used cut-head/fill-in-tail and cut-tail/fill-in-head… Outline Problem Positions 5 -6 as cut points to cut-head/fill-in-tail Related Works Genetic Algo Pr 2 R 3 R 4 R 1 R 5 R 8 R 6 R 7 R 9 Our Approach Pr 3 R 5 R 8 R 1 R 3 R 7 R 2 R 9 R 4 R 6 Pr 2’ R 2 R 3 R 4 R 1 R 5 R 6 R 7 R 8 R 9 Case Study Results Discussions Conclusions - variation allows searching out different available niches, find better fitness values and subsequently better solutions - never produce chromosomes containing duplicate genes.

GA Mutation Operator Using Interactive Genetic Algorithm for Requirements Prioritization Mutation is basically a swap operator, we used requirement-pair-swap Outline Problem Related Works Pr 2 Genetic Algo R 2 R 3 R 4 R 1 R 5 R 8 R 6 R 7 R 9 R 2 R 3 R 6 R 1 R 5 R 8 R 4 R 7 R 9 Our Approach Case Study Results Pr 2’ Discussions Conclusions - mutation causes movement in the search space - may produce a stronger chromosome.

GA Selection Operator Using Interactive Genetic Algorithm for Requirements Prioritization We used the Tournament Selection for our approach Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions - allows the selection pressure to be easily adjusted - faster than other selection operators i. e. Roulette Wheel Selection - better convergence

Our IGA Approach Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions 1. 2. 3. Acquisition and coding of set of Requirements and Domain Knowledge Apply Interactive Genetic Algorithm (exploiting User Knowledge) Output of the ranking (the most promising individual)

Our Approach: Pairs to be Elicited Using Interactive Genetic Algorithm for Requirements Prioritization Indv. ID Individuals Disagree Pr 1 < R 1, R 3, R 2, R 4, R 5, R 6, R 7, R 8, R 9 > 8 Pr 2 < R 2, R 3, R 4, R 1, R 5, R 8, R 6, R 7, R 9 > 8 Pr 10 < R 1, R 4, R 2, R 3, R 5, R 6, R 9, R 8, R 7 > 13 Pr 4 < R 4, R 5, R 6, R 3, R 2, R 1, R 8, R 9, R 7 > 15 Outline Pr 3 < R 5, R 2, R 1, R 3, R 7, R 8, R 6, R 9, R 4 > 16 Problem Pr 9 < R 1, R 3, R 5, R 7, R 9, R 2, R 4, R 6, R 8 > 17 Pr 5 < R 7, R 8, R 6, R 5, R 2, R 3, R 4, R 9, R 1 > 23 Pr 8 < R 8, R 9, R 6, R 7, R 4, R 5, R 2, R 3, R 1 > 29 Genetic Algo Pr 6 < R 5, R 6, R 7, R 8, R 9, R 1, R 2, R 3, R 4 > 29 Our Approach Pr 7 < R 9, R 8, R 7, R 6, R 5, R 4, R 3, R 2, R 1 > 30 Related Works Case Study Pr 1 = < R 1, R 3, R 2, R 4, R 5, R 6, R 7, R 8, R 9 > Results vs Discussions Pr 2 = < R 2, R 3, R 4, R 1, R 5, R 8, R 6, R 7, R 9 > Conclusions Ranked individuals w. r. t. disagreement TIE PAIRS Pr 1, Pr 2 (R 1, R 3), (R 2, R 3), (R 6, R 8), (R 7, R 8) Candidate pairs to be asked to the decision maker

Our Approach: User Feedback Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions TIE PAIRS Pr 1, Pr 2 (R 1, R 3), (R 2, R 3), (R 6, R 8), (R 7, R 8) Prio Why ties appear in population? (i) Contradictory information w. r. t. initial constraints (ii) Nothing is expressed explicitly in initial constraints (iii) Common Domain Knowledge but different positions in the individuals Simple example: Why (R 7, R 8)? Case I: Contradictory w. r. t. Prio & Dep. . Why (R 2, R 3)? Case III: Common knowledge but different positions Dep

Our Approach: User Feedback Using Interactive Genetic Algorithm for Requirements Prioritization TIE PAIRS Pr 1, Pr 2 (R 1, R 3), (R 2, R 3), (R 6, R 8), (R 7, R 8) Outline R 1 < ? > R 3 R 2 < ? > R 3 R 6 < ? > R 8 Problem Related Works Genetic Algo Our Approach Case Study Experience & knowledge ‘<‘ or ‘>’ Eli User Preference Graph eli. Ord Results Discussions Conclusions So, user knowledge is playing important role…

Our Approach: New Round with New Constraints Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem § The new evolved population after using GA operators on population: ü Crossover ü Mutation ü Selection § is compared against the new set of constraints graphs Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Prio Dep Eli

Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Evaluation of the Approach

The Case Study Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions § Prioritize requirements for a real software system, as part of the project ACube (Ambient Aware Assistance) designing a highly technological monitoring environment to be deployed in nursing homes to support medical and assistance staff § After user requirements analysis phase, 60 user requirements and 49 technical requirements Four macro-scenarios have been identified. ID Macro-Scenario # of requirements FALL Monitoring falls 26 ESC Monitoring escapes 23 Monitoring dangerous behavior 21 The three scenarios 49 MON ALL Conclusions *ACube is a social welfare project coordinated by Fondazione Bruno Kessler (FBK) and funded by Autonomous Province of Trento under Bando Grandi Progetti, 2006.

Gold Standard (GS) Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem § For each of the four macro-scenarios, we obtained the Gold Standard (GS) prioritization from the Software Architect of the ACube project ü The GS prioritization is the ordering given by the software architect to the requirements when planning their implementation during the ACube project. Related Works Genetic Algo Our Approach Why Gold Standard? Case Study Results Discussions Conclusions ü To measure disagreements with respect to GS. ü To evaluate our approach in terms of disagreement against other non-interactive approaches using the same GS.

Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Research Questions

RQ 1 Using Interactive Genetic Algorithm for Requirements Prioritization Outline RQ 1 (Convergence) Can we observe convergence with respect to the finally elicited fitness function? - Convergence is not obvious immediately, as Eli Graph is evolving at early stages. - Although the full fitness function is known only at end of elicitation process. The best individual in each population converges toward a low value of the final fitness function. Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions YES!

RQ 2 Using Interactive Genetic Algorithm for Requirements Prioritization RQ 2 (Role of interaction) Does IGA produce improved prioritizations compared to non-interactive requirement ordering? Outline Problem Related Works YES! Genetic Algo Our Approach Case Study Results Discussions Conclusions IGA outperforms substantially GA (and RAND), especially when a higher number of pairwise comparisons can be carried out

RQ 3 Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem RQ 3 (Role of initial precedence constraints) How does initial availability of precedence constraints affect the performance of IGA? Improves! Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions - Different type of Domain Knowledge affects IGA significantly - The improvement of IGA over GA is even higher when limited ranking information is available

RQ 4 Using Interactive Genetic Algorithm for Requirements Prioritization Outline RQ 4 (Robustness) Is IGA robust with respect to errors committed by the user during the elicitation of pairwise comparisons? YES! Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions IGA can tolerate user errors up to 20%.

General Discussion Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions § Cost/benefit trade offered by IGA as compared to AHP is extremely interesting ü With an elicitation effort reduced to 10% of the one required by AHP, IGA produces an apprx. ordering which has a quite low disagreement from the requirement positions in the GS. § User Errors tolerance offered by IGA as compared to AHP is highly reasonable ü With an elicitation of very significant amount of less pairs than AHP (i. e. pairwise comparisons), it is reasonable even if IGA accepts up to 20% user error, while using AHP even 10% user error can result a total bad ordering.

Conclusions & Future Works Using Interactive Genetic Algorithm for Requirements Prioritization q We proposed an Interactive Genetic Algorithm to collect pairwise information useful to prioritize the requirements for a software system. q We also verified the robustness of the algorithm with respect to increasing user feedback errors. Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions q We evaluated the approach in a real project (ACube). q In summary, we contributed A NOVEL APPROACH to prioritize requirements & tested its effectiveness empirically. What’s Next? Algorithm: - refining the algorithm - improving GA operators Experiment - off-line: comparisons with other approaches - on-line: controlled experiments with real object (i. e. human)

Paper Publication Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions A paper was published from this work and was presented at SSBSE 2010: Paolo Tonella 1, Angelo Susi 1, and Francis Palma 2, "Using Interactive GA for Requirements Prioritization" in 2 nd International Symposium on Search Based Software Engineering 2010. 1 Fondazione Bruno Kessler, Software Engineering Research Unit; 2 Department of Inf. Eng. and Computer Science, University of Trento.

Using Interactive Genetic Algorithm for Requirements Prioritization Outline Problem Related Works Genetic Algo Our Approach Case Study Results Discussions Conclusions Thank You!