A Pragmatic View of Formal Methods The HiLite

A Pragmatic View of Formal Methods The Hi-Lite Project Robert B K Dewar President & CEO, Ada. Core Emeritus Professor of Computer Science, New York University SSS ‘ 11 Feb 10 th 2011 www. adacore. com

An observation from Yogi Berra • In theory there is no difference between theory and practice. In practice there is. • Variously attributed to Yogi Berra, Einstein etc. Copyright © 2009 Ada. Core Slide: 2

Copyright © 2009 Ada. Core Slide: 3

Correctness and Reliability • Not really the same thing • Often merged in people’s thinking • The issue of large reliable programs • Correctness means adherence to some formal model • Reliability means that a program works • Many reasons why a correct program might be unreliable. • And a reliable program may well be incorrect Copyright © 2009 Ada. Core Slide: 4

We can do Pretty Well in Practice • No one has ever died on a commercial airplane as a result of a software bug. • We have however had some hair raising close calls • But a close call is a close call • When it comes to avionics, software is not the weak link in the chain. • But we could do better • And most certainly best practices are not widely applied – Cars – Medical Equipment – Critical financial applications Copyright © 2009 Ada. Core Slide: 5

Safety And Security • Safety means we don’t have bugs that kill people • Security means bad guys can’t cause chaos • We have lots of bad guys around who know a lot • Any safety critical system is security critical now • How vulnerable are we • TV thinks we are very vulnerable – Alias and Avionics systems – 24 and Air Traffic Control • Don’t be too quick to dismiss fiction – Clancy crashed a plane long before 9/11 Copyright © 2009 Ada. Core Slide: 6

Proving Entire Programs Correct • A nice conceptual goal • But definitely a case where theory and practice are not on the same page • Fundamental problems stand in the way Copyright © 2009 Ada. Core Slide: 7

Producing a Full Formal Specification • No use unless we find an appropriate level of abstraction. • Every program is a full formal specification in some sense, but that’s not useful • Every generation of programming languages has been sold as high level specification eliminating the need for programmers – Fortran, mathematicians will enter formulas directly – COBOL, people will stick business models into the computer directly – 4 G languages, no, honestly, they really will stick them in directly – Modeling languages, no more need for expensive programmers • Expensive, but might be worth the cost if it was a panacea, but it is not. Copyright © 2009 Ada. Core Slide: 8

Difficulties in Producing Full Specifications • No one understands formal specifications • There are things that can’t be formally specified – Good error messages in a compiler – Intuitive user interfaces • Deliberate incompleteness • Difficulties of proving termination • Difficulties of specifying programming languages • Anyway, they will have errors • A specification is basically a large program • And often one that has never been tested Copyright © 2009 Ada. Core Slide: 9

Proving Properties of Programs • Definitely a promising approach • Not just promises, we can deliver – Freedom from run-time errors – Proof of security properties • Security a particular important target – Secure programs do not have to work – But they do have to be secure Copyright © 2009 Ada. Core Slide: 10

The Role of Testing • Right now, we largely rely on testing, e. g. in DO-178 B • Thorough testing definitely improves our confidence • But testing is fundamentally incomplete • DO-178 B has no formal model, it just works • So can we eliminate testing? • Almost no one thinks we can eliminate 100% of testing • I don’t care how much you have proved, I don’t want to fly on a plane with avionics that has never been tested. • But if we can’t eliminate ANY testing, then bleak times ahead formal methods, certainly in the safety area. • So the idea (enshrined in DO-178 C) is to figure out how to give some credit formal methods, eliminating some testing requirements. Copyright © 2009 Ada. Core Slide: 11

Proofs Replacing Testing • Specific Unit tests might be replaced by proofs, e. g. of the correctness of an algorithm, eliminating the need for complete domain testing. • Object Oriented Programs and Dynamic Dispatch • Thorough testing impractical (every call to every method). • Require LSP in OO hierarchies • OK, we can require it, but how to verify? • Unit testing? ? ? • This is a place where proof techniques do make sense Copyright © 2009 Ada. Core Slide: 12

Reasoning About Existing Programs • The use of static analysis • Finding bugs is useful, but not so useful • Guaranteeing absence of bugs is definitely useful • But eliminating all false negatives can generate too many false positives • Helps to start with an appropriate language, e. g. in Ada, range declarations add a focus for analysis. • The Microsoft Vista story – Added 500, 000 annotations – Found 100, 000 potential buffer overflows • Finding pre and post conditions • Incremental Analysis Copyright © 2009 Ada. Core Slide: 13

Putting it All Together, the Hi-Lite Project • In practice we will make use of a combination of techniques and tools – Proof of correctness and formal specification – Thorough testing – Use of static analysis tools • All these tools can benefit from the addition of annotations about what should be going on. – Postconditions – Preconditions – Invariants • But different tools have different views – Is a precondition something to be proved? – Or is it something to be tested? – Or is it there to give hints to a static analyzer Copyright © 2009 Ada. Core Slide: 14

Preconditions and Postconditions • At one level, they are just formalizations of comments. Comments are a good thing if – They are precise – They are complete – They are kept up to date • But of date or incorrect comments are a menace • So formalizing them can only help • Not that much work once you get used to it • But what language should be used • Ada 2012 adds extensive capabilities for annotations including preconditions, postconditions, invariants, and predicates. Copyright © 2009 Ada. Core Slide: 15

Trying to Unify the Language Requirements • If annotations/contracts are useful for proof, testing, and static analysis, how about we use the same annotation language, so we can integrate the different approaches within a common framework? • Multiple approaches are problematic • In Ada. Core technology, four syntaxes for preconditions – Precondition pragma for Ada 95/2005 – Pre aspect in Ada 2012 – SPARK annotation for preconditions – Code. Peer syntax for discovered precondition Copyright © 2009 Ada. Core Slide: 16

Hi-Lite • One important goal of Hi-Lite is to create a unified language for annotations that can be used in all contexts. • Then the second step is to create a unified development environment in which different tools can use the same source base in different contexts. • The goal: to facilitate (i. e. make cheaper) the integration of formal methods and testing regimes. • We need to create industrial workflows that incorporate verification on a routine basis • These tools need to integrate different programming approaches as well (different languages, modeling …) Copyright © 2009 Ada. Core Slide: 17

Problems in the Unification • The requirements for mathematical specifications and programming language expressions are different. • Does A+B mean A+B or does it mean apply some weird programming notion of + with overflows or wrap around? – with Pre => A + B <= Integer’Last • Does A or B mean A or B, or does it mean do some strange evaluation in which an overflow in B causes an exception even if A is True? • Not insuperable, but such issues stand in the way of this being a simple job of getting people to agree on syntax. Copyright © 2009 Ada. Core Slide: 18

Conclusion • We have a lot of industrial experience to feed in here • We have creators of tools and users very interested in participating in creating a workable framework – Ada. Core – Praxis – CEA – INRIA • We have mportant industrial partners who need this technology now, so this is much more than just research. – Altran – Astrium – Thales – Rockwell Copyright © 2009 Ada. Core Slide: 19