Managing DesignTime Uncertainty Michalis Famelis Marsha Chechik MODELS

Managing Design-Time Uncertainty Michalis Famelis, Marsha Chechik MODELS 2017 Austin TX, USA 1

Uncertainty in Software Development Many design alternatives Incomplete information Conflicting stakeholder opinions Uncertainty during the design of software. 2

Uncertainty in: Environment Design-time What conditions will the system operate in? What should the system be like? Main concern: adapting to change Main concern: making design decisions Mitigated by uncertainty-aware software development methodology 3

Management of Design-Time Uncertainty Key development goals: Quality Speed (time to market) What can developers do? Make a provisional decision and “run with it” Wait until uncertainty gets resolved Fork and maintain a set of solutions We propose: Defer resolution of uncertainty but incorporate uncertainty handling into the development process to allow progress 4

Articulation of uncertainty Outline • Partial Models: • Semantics • Notation Deferral of decisions • Lifting: • Verification • Diagnosis • Transformation Resolution of uncertainty • Methodology and Tool Support • Worked-out Examples • Conclusion, Future Work • Refinement • Decision-making 5

Articulation Outline • Partial Models: • Semantics • Notation Deferral • Lifting: • Verification • Diagnosis • Transformation Resolution • Methodology and Tool Support • Worked-out Examples • Conclusion, Future Work • Refinement • Decision-making 6

Design-time Uncertainty Knowns Possibilities Design decisions assumed known Known Unknowns Alternative solutions assumed elicited Unknowns US Secretary of Defense, Donald Rumsfeld discusses Iraqi WMDs, February 12, 2002 7

Representing Uncertainty with Semantics: Partial Models Solver. Exception Solver. Exception May element Propositional variable: Unsure if it should “the exists” be anelement inner class. x + effect : String Solver. Exception Unsure if we need this field. Xv. Y x=T, y=F Solver. Exception y x=F, y=F + effect : String x=F, y=T May formula Solver. Exception x=T, y=T Concretization [ICSE’ 12] Partial Models: Towards Modeling and Reasoning with Uncertainty, M. Famelis, R. Salay and M. Chechik 8

Articulation Outline • Partial Models: • Semantics • Notation Deferral • Lifting: • Verification • Diagnosis • Transformation Resolution • Methodology and Tool Support • Worked-out Examples • Conclusion, Future Work • Refinement • Decision-making 9

Refinement: Reduce the Set Solver x Operationally: make a decision Solver. Exception x=F, y=F Solver. Exception + effect : String y (X v Y) Λ ¬(X Λ Y) Solver. Exception + effect : String x=F, y=T Declaratively: with a property [ICSE’ 12] Partial Models: Towards Modeling and Reasoning with Uncertainty, M. Famelis, R. Salay and M. Chechik x=T, y=F Solver. Exception + effect : String x=T, y=T 10

Articulation Outline • Partial Models: • Semantics • Notation Deferral • Lifting: • Verification • Diagnosis • Transformation Resolution • Methodology and Tool Support • Worked-out Examples • Conclusion, Future Work • Refinement • Decision-making 11

Deferring Uncertainty Resolution DOES NOT COMPUTE MAKE DECISIONS FIRST Automated Procedure 12

Deferral Through “Lifting” Verification: thorough checking. Transformations: preservation of set of concretizations. Development can continue with Automated Procedure Adapting automation to work with partial models 13

Lifting Verification Example property: “Every inner class has at least one attribute” Property holds for… …all concretizations …some but not all + effect : String Lifted SAT-based verification …none • Applies directly to the partial model • Does not enumerate concretizations • Computes result using three-valued logic [ICSE’ 12] Partial Models: Towards Modeling and Reasoning with Uncertainty, M. Famelis, R. Salay and M. Chechik 14

Lifting Transformations ? [MODELS 2013] Transformation of Models Containing Uncertainty, M. Famelis, R. Salay, A. Di Sandro and M. Chechik 15

Lifting Transformations ? [MODELS 2013] Transformation of Models Containing Uncertainty, M. Famelis, R. Salay, A. Di Sandro and M. Chechik 16

Lifting Transformations [MODELS 2013] Transformation of Models Containing Uncertainty, M. Famelis, R. Salay, A. Di Sandro and M. Chechik 17

Lifting Transformations Design decisions not affected Neither is the transformation! Step 1: Determine applicability Step 2: Transform graph Step 3: Transform constraints [MODELS 2013] Transformation of Models Containing Uncertainty, M. Famelis, R. Salay, A. Di Sandro and M. Chechik 18

Articulation Outline • Partial Models: • Semantics • Notation Deferral • Lifting: • Verification • Diagnosis • Transformation Resolution • Methodology and Tool Support • Worked-out Examples • Conclusion, Future Work • Refinement • Decision-making 19

UMLet Bug #10 Articulation K 0 Sequence Diagram that fixes the bug but violates consistency constraints Make Partial U 1: Bugfix does not affect the Paste command U 2: Consistency fix K 1 preserves bugfix do. Work Partial model of all ways to fix consistency. K 1 Deferral True U 11)) ((U VVeerriiffyy VVeerrififyy ((UU 22)) new. Information Maybe 115 elements, 44 concretizations Observed speedups consistent with experiments. Resolution )) e((UU 22 e l l p p m am rr. EExxa e e t t n n CCoouu Property used to do refinement K 1 Constrain(U 2) K 3 K 2 Decide K 4 20

Uncertainty Lifecycle Management Articulation of uncertainty degree of Uncertainty Deferral of decisions Time • Degree of uncertainty: size of the set of concretizations Resolution of uncertainty • Ultimately, a single concrete model: all uncertainty resolved 21

Design-Time Uncertainty Management (De. TUM) model Articulation of uncertainty Triggered by developer. Factors: • presence/absence of information • work that needs to be done do. Work new. Uncertainty Deferral of decisions new. Information Don’t have to necessarily do engineering tasks to transition Also: Undo Might encounter new points of uncertainty more Uncertainty more. Information Gradual removal of uncertainty more. Work Resolution of uncertainty 22

Uncertainty Management Operators • • • Articulation of uncertainty • • • Decide Constrain Generate Counter Example Generate Diagnostic Core do. Work new. Uncertainty • • • Deferral of decisions new. Information more Uncertainty more. Information Construct Make. Partial Expand Transform Verify Deconstruct more. Work Resolution of uncertainty 23

Example Operator Specification Name Construct Description Create a partial model from a given set of concrete models that are alternative resolutions to uncertainty. Inputs A set of non-partial models. Outputs A partial model. Usage context The developer has at their disposal a known, fully enumerated set of alternative models, but has insufficient information about which of the models is best suited for their purpose. Preconditions No partial model exists. The set of models must be known and fully enumerated. Postconditions The resulting partial model is in Graphical Reduced Form (GRF) and its set of concretizations is exactly the set of input models. Limitations The developer must have the full knowledge of the input set. Implementation Described in [ICSE’ 12] as operator “OP 1: Construction”. [ICSE’ 12] Partial Models: Towards Modeling and Reasoning with Uncertainty, M. Famelis, R. Salay and M. Chechik 24

MU-MMINT (pronounced “moomin”) Decision Tree Editor Partial Model Editor Dashboard & Traceability Verification & Refinement Support Lifted Transformations MMINT: “Model Management INTeractive” Eclipse Z 3 SMT Solver Henshin Graph Transformation Engine MU-MMINT demo: https: //youtu. be/k. AWUm-i. Fat. M MMINT demo: https: //youtu. be/7 B 7 Yu. V-Jvrc Available at https: //github. com/adisandro/MMINT [ICSE 2015] MU-MMINT: an IDE for Model Uncertainty, M. Famelis, N. Ben-David, A. Di Sandro, R. Salay, and M. Chechik [MODELS’ 15] MMINT: A Graphical Tool for Interactive Model Management, A. Di Sandro, R. Salay, M. Famelis, S. Kokaly, and M. Chechik, 25

Articulation Outline • Partial Models: • Semantics • Notation Deferral • Lifting: • Verification • Diagnosis • Transformation Resolution • Methodology and Tool Support • Worked-out Examples • Conclusion, Future Work • Refinement • Decision-making 26

Metamodel to Relational Schema Scenario: create a metamodel for Petri nets, then create a schema for storing them Place Token Transition • Atlas Metamodel zoo: 8 different designs / 5 design decisions • Partial model N 0 created using MU-MMINT • • • Demo partial model editor Demo Verification and Diagnosis Demo Transformation 27

Petri Net Metamodel Additional uncertainty: 76 elements, 18 concretizations Articulation/1 Design Decisions Which domain-specific extensions should the metamodel support? 1: do. Work N 0 Construct New partial model with 117 elements, 360 concretizations. Articulation/2 N 2 Expand 3: more. Uncertainty N 3 Partial relational schema model, 192 elements U 3: Diagram element locations are stored Deferral/1 2 sec 4: do. Work Object. Relational Mapping transformation with 5 layered Henshin rules N 0 Transform N 1 Verify(U 3) 114 sec N 3 Transform N 4 Maybe Deferral/2 Partial relational schema model, 293 elements 2: new. Info Resolution/1 N 0 Decide N 2 Resolution/2 5: new. Info N 3 Decide N 5 28

Lessons Learned from Worked Examples Must better support Articulation with automation Stages of DETUM not rigid (Verification/Diagnosis) May formula makes engineering of lifting hard Changing modality of properties may be more appropriate response to bad verification result 29

Articulation Outline • Partial Models: • Semantics • Notation Deferral • Lifting: • Verification • Diagnosis • Transformation Resolution • Methodology and Tool Support • Worked-out Examples • Conclusion, Future Work • Refinement • Decision-making 30

Managing of Design-Time Uncertainty Defer resolution of uncertainty but incorporate uncertainty handling into the development process to allow progress Articulation of uncertainty Deferral of decisions Resolution of uncertainty • Partial Models: • Semantics • Notation • Lifting: • Verification • Diagnosis • Transformation • Refinement • Decision-making • DETUM model • Uncertainty Management Ops • MU-MMINT 31

Future Work Relax underlying assumptions Design decisions known; alternatives elicited Better support uncertainty articulation Leverage development context Systematically elicit design options Combine with existing methodologies (e. g. Scrum, Kanban) 32

Managing of Design-Time Uncertainty Michalis Famelis, Marsha Chechik Articulation of uncertainty Deferral of decisions Resolution • Partial Models: • Semantics Que • Notation stio • Lifting: ns? • Verification • Diagnosis • Transformation • Refinement • Decision-making • Methodology and Tool Support • Worked-out Examples • Discussion, Future Work of uncertainty 33