CSC 2108 Automated Verification or Everything you Wanted

CSC 2108: Automated Verification or Everything you Wanted to Know about Model-Checking Ü Instructor: Marsha Chechik Ü Contact: chechik@cs. toronto. edu Bahen 5236, 416 -978 -3820 Ü TA: Arie Gurfinkel arie@cs. toronto. edu Ü Web page: http: //www. cs. toronto. edu/~chechik/courses 05/csc 2108

Software Engineering and Formal Methods

Motivation “To err is human but to really foul things up requires a computer” Farmer’s Almanac for 1978 “Capsules of Wisdom” “ 60% of all major industrial disasters from 1921 to 1989 occurred after 1975” Nancy Leveson “Safeware: Computers and Technology”

Example: auto-pilot Problem: “Design a part in auto-pilot that avoids collision with other planes. ” Solution: “When distance is 1 km, give warning to other plane and notify pilot. When distance is 300 m, and no changes in the course of other plane were noticed, go up to avoid collision”

Problem with solution Ü Both planes have the same software. Both go up. . .

This happens in real software! Ü Some famous bugs ÄNASA Space Rover, Intel floating point processor, etc. Ü Hard to predict all behaviours! ÄUS aircraft went to southern hemisphere and … flipped when crossing the equator ÄAir traffic controller: US to Britain. Ø It never dealt with problem of 0 degrees longitude. Ø Result: software “folded” Britain along Greenwich Meridian ÄSoftware written for US F-16 Ø accidents when reused in Israeli aircraft flown over the Dear Sea (altitude < sea level) ÄYear 2000 problem

Yet more such examples Ü NASA Space Shuttle software (in use since 1980) Ä16 severity-level 1 software errors Ä8 remained in code that was used in flights Änone encountered during flights Ätotal size - only 400, 000 words

So, why not test? Testing only shows presence of bugs not their absence!

Formal Methods Mathematically-based techniques for describing properties of systems Ü Provide framework for ÄSpecifying systems (and thus notion of correctness) ÄDeveloping systems ÄVerifying correctness Ø Of implementation w. r. t. the specification Ø Equivalence of different implementations Ü Reasoning is based on logic ÄAmenable to machine analysis and manipulation ÄIn principle, can verify everything that is true in the system! Ø Given enough time, skill and patience

Why aren’t FMs used more? “Formal methods can revolutionize development!” “Formal methods are difficult, expensive, not widely useful and for safety-critical systems only”

… and one more problem Ü Need to know what to build (specification) before you start building System engineering Analysis Design Coding “water-fall” model Testing Maintenance Ü Unrealistic! ÄMay need to discover what to build iteratively ÄSoftware changes all the time

Formal Methods “Light” Ü Partial application of formal methods Äonly parts of systems are specified Ü Emphasis on analysis of some properties Äsecurity, fairness, deadlock freedom, rather than complete verification Ü Debugging rather than assurance Ü Automation Most successful lightweight technique: Model-Checking

Model-Checking

Overview of Automated Verification SW/HW artifact abstraction Correct? Correctness properties Model Extraction Translation Model of System Temporal logic Checker Engine Answer + Counter-example

CTL Model-Checking Ü CTL: Branching-time propositional temporal logic Ü Model - a tree of computation paths Ü Example: S 1 S 2 S 1 S 3 Kripke Structure S 2 S 1 S 3 S 1 S 2 S 3 S 1 S 3 Tree of computation

Models: Kripke Structures Ü Conventional state machines ÄM = <S, A, s 0, I , R> ÄS is a (finite) set of states s 0 p p q r ÄA is a (finite) set of propositional variables Äs 0 is a unique initial state (s 0 S) q r p q s 1 s 2 r ÄI: S 2 A is a labeling function that maps each state to the set of propositional variables that hold in it ÄR S S is a (total) transition relation

Propositional Variables ÄFixed set of atomic propositions {p, q, r} ÄAtomic descriptions of a system Ø“Printer is busy” Ø“There are currently no requested jobs for the printer” Ø“Conveyer belt is stopped” ÄHow to choose them? ÄShould not involve time!

CTL: Computation Tree Logic propositional temporal logic. allows explicit quantification over possible futures Ü Syntax: True ( ) and False ( ) are CTL formulae; propositional variables are CTL formulae; if and are CTL formulae, then so are: , , EX --- holds in some next states; EF --- along some path, is true in a future state; E[ U ] --- along some path, holds until holds; EG --- along some path, holds in every state ÄUniversal quantification: AX , AF , A[ U ], AG

Examples EX (exists next) AX (all next) EG (exists global) AG (all global)

Examples, Cont’d EF (exists future) AF (all future) EU (exists until) AU (all until)

CTL (Cont’d) Ü Examples: ÄProperties that hold: Ø (EX p)(s 0) Ø (A[p U q])(s 0) Ø ( EX AF p )(s 0) Ä Properties that fail: Ø (A[ p U q])(s 0) s 0 p q r s 2 p q r s 1

Some Statements To Express ÄIt is possible to get to a state where started holds, but ready does not hold Ø EF (started ready) ÄWhen a request occurs, it will eventually be acknowledged Ø AG (request AF acknowledge)

CTL Model-Checking Ü Receive: ÄKripke structure K ÄTemporal logic formula Ü Assumptions: ÄFinite number of processes Ø Each having a finite number of finite-valued variables ÄFinite length of a CTL formula Ü Algorithm: ÄLabel states of K with subformulas of that are satisfied there and working outwards towards . ÄOutput states labeled with Example: EX AG (p E[p U q])

CTL Model-Checking (Cont’d) EX AF ÄLabel any state with EX if any of its successors are labeled with ÄIf any state s is labeled with , label it with AF ÄRepeat: label any state with AF if all of its successors are labeled with AF EX until there is no change , AF , AF AF , AF , AF , AF

Counterexamples Ü Explain: ÄWhy the property fails to hold Äto disprove that holds on all elements of S, produce a single element s S s. t. holds on s. Ø counterexamples restricted to universally-quantified formulas Ø counterexamples are paths (trees) from initial state illustrating the failure of property ÄAG p s 0 p p q r s 2 p q r s 1 p p q r s 1 ÄAX p AX q p s 0 q r p s 1 q r s 2 p q r

Symbolic model-checking Ü Idea of model-checking x Ärecursively go through the structure of the. F CTL property… Äassociating each subformula with a set of states where each subproperty is true Ü Symbolic model-checking Äeffective cure for state explosion problem Äuse symbolic representation for sets of states Äuse symbolic representation for transition relation Äuse binary decision diagrams (BDDs) to encode these Ü Example: Ä x y in classical logic T y F T

Questions: Ü Where does the Kripke structure come from? ÄComputing it from several concurrent processes Ø Models of concurrency: maximum parallelism, interleaving ÄWhat happens if Kripke structure is infinite? Ø Abstraction! ÄHow to represent large Kripke structures? Ø Abstraction! Ø Symbolic encoding of transition relation Ø SAT-based encoding (BMC) Ø Symmetry (not part of the course) Ø Compositional verification Ü What other methods are there?

2108: Automated Verification A course on model-checking technology Ü Theory: ÄDifferent temporal logics (CTL, LTL, mu-calculus) ÄFoundations: Ø Automata theory (automata on infinite strings, tree automata, alternating tree automata), lattice theory, fixpoint computations Ø Introduced as needed! ÄDifferent model-checking approaches: Ø Explicit-state, Symbolic, Bounded (BMC) Ø Game-theoretic Ø Automata-theoretic Ü Tools: ÄSMV, SPIN, Concurrency workbench

Automated Verification (Cont’d) Ü Making this work: ÄDecision diagrams, SAT solvers ÄCompositional verification ÄPartial order reduction ÄAbstraction and refinement Ü Some applications ÄDebugging ÄRequirements monitoring ÄTest case generation Ü Extensions ÄQuery-checking ÄModel-checking over partial and inconsistent systems ÄEtc.

2108: Pragmatics Ü Time: Tuesdays, 2 -5 p. m. Ü Readings: ÄBook: Ø Clarke, Grumberg, Peled. Model-Checking. MIT Press, 1999 ÄMany research papers Ü Course work: Ä2 -3 small modeling exercises ÄRegular (small) paper-and-pencil homeworks ÄResearch project ÄPossibly presentation of a topic related to model-checking Ü Prerequisites: ÄDiscrete mathematics, general inclination towards theory ÄPrior experience with concurrent systems a plus

Pros and Cons of Model-Checking Ü Often cannot express full requirements ÄInstead check several smaller properties Ü Few systems can be checked directly ÄMust generally abstract Ü Work better for certain types of problems ÄVery useful for control-centered concurrent systems Ø Avionics software Ø Hardware Ø Communication protocols ÄNot very good at data-centered systems Ø User interfaces, databases

Pros and Cons, Cont’d Ü Largely Ü Better automatic and fast use for debugging Ä… rather than assurance Ü Testing vs model-checking ÄUsually, find more problems by exploring all behaviours of a downscaled system than by testing some behaviours of the full system

Readings Ü List of readings on the web site ÄChapter 1 of the textbook ÄSeveral papers on formal methods in general. Links from the reading list on the course web page Ü For next week: ÄChapter 3 of the textbook