Solidifying Software Interfaces Checkable Contracts Thomas Ball Testing

Solidifying Software Interfaces: Checkable Contracts Thomas Ball Testing, Verification and Measurement Microsoft Research http: //research. microsoft. com/~tball/

The. NET Framework Richness A Once in a Decade Change s t n one p Com Windows 3. 0 s API COM Win 32 Win 16 1980 1990 MFC 2000 s e c i v Ser

Trustworthy Commitment • Microsoft Cultural Shift – Thousands of hours spent in security reviews on. NET Framework to date – Foundstone, @Stake security reviews • “Hardening” the. NET Framework • Making Security Easier for Customers – Prescriptive Architectural Guidance – Feature changes in. NET Framework

Tools . NET Framework Client Application Model Windows Forms System. Windows. Form s Web & Service Application Model Data Systems Application Model Yukon Compact Framework System. Data. Sql. Server System. Windows. Forms ASP. NET System. Web Mobile PC & Devices Application Model Command Line System. Console NT Service System. Service. Proces s Presentation Communication Data System. Messaging System. Data System. Web. UI Sql. Client Data. Set Page Web. Controls Control Adaptors Sql. Types Mapping Html. Controls Design Sql. XML Object. Spaces Mobile. Controls System. Windows. Forms System. Directory. Services System. Runtime. Remotin g System. Web. Service s Web. Service Odbc. Client Object. Space Description Ole. Db. Client Query Discovery Oracle. Client Schema Protocols Forms System. Xml Control Schema Print Dialog Serializatio n Design System. Web Xpath Query Personalization Caching Session. State System. Drawing System. Net Http. Web. Request Network. Information Ftp. Web. Listener Sockets Ssl. Client. Stream Cache Web. Client Fundamentals Security Base & Application Services System. Timers System. Text System. Globalization System. Design System. Serialization System. IO System. Threading System. Collections Compiler. Services System. Reflection Interop. Services System. Web. Configuration System. Security Access. Control System. Component. Model System. Code. Dom Serialization Configuration Generic Ports System. Runtime System. Web. Security System. Enterprise. Services System. Transactions Permissions Deployment/Management System. Web System. Configuration Administration System. Resources Management Credentials Policy Cryptography Principal System. Management Token System. Deployment System. Diagnostics

Interfaces Everywhere! Client API Implementation But no contracts!

Microsoft Powerpoint EULA Point 11 • 11. EXCLUSION OF INCIDENTAL, CONSEQUENTIAL AND CERTAIN OTHER DAMAGES. TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL MICROSOFT OR ITS SUPPLIERS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT, OR CONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING, BUT NOT LIMITED TO, DAMAGES FOR LOSS OF PROFITS OR CONFIDENTIAL OR OTHER INFORMATION, FOR BUSINESS INTERRUPTION, FOR PERSONAL INJURY, FOR LOSS OF PRIVACY, FOR FAILURE TO MEET ANY DUTY INCLUDING OF GOOD FAITH OR OF REASONABLE CARE, FOR NEGLIGENCE, AND FOR ANY OTHER PECUNIARY OR OTHER LOSS WHATSOEVER) ARISING OUT OF OR IN ANY WAY RELATED TO THE USE OF OR INABILITY TO USE THE SOFTWARE PRODUCT, , THE PROVISION OF OR FAILURE TO PROVIDE SUPPORT SERVICES, OR OTHERWISE UNDER OR IN CONNECTION WITH ANY PROVISION OF THIS EULA, EVEN IN THE EVENT OF THE FAULT, TORT (INCLUDING NEGLIGENCE), STRICT LIABILITY, BREACH OF CONTRACT OR BREACH OF WARRANTY OF MICROSOFT OR ANY SUPPLIER, AND EVEN IF MICROSOFT OR ANY SUPPLIER HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

The GPL • • 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Is There any Program That Satisfies Its Contract?

Informal Contract: Sockets the "communication domain" in which communication is to take place; see protocols(5). Sockets of type SOCK_STREAM are full-duplex byte streams, similar to pipes. A stream socket must be in a connected state before any data may be sent or received on it. A connection to another socket is created with a connect(2) call. Once connected, data may be transferred using read(2 V) and write(2 V) calls or some variant of the send(2) and recv(2) calls. When a session has been completed a close(2 V), may be performed. Out-of-band data may also be transmitted as described in send(2) and received as described in recv(2). The communications protocols used to implement a SOCK_STREAM insure that data is not lost or duplicated. If a piece of

What is an API Contract? • Pre-conditions – the conditions a client must establish before calling an API – “A filehandle must be in an open state before you call fread” • Post-conditions – the conditions an implementation (of an API) must establish upon its termination – “If the file is present, fopen returns a filehandle in the open state”

Formalizing Contracts • Pre/post conditions – Eiffel: “design by contract”, integrated into language – JML: pre/post language (in comments) • Monitors – security automata – SLIC - SLAM’s API rule language • Models – ASML: separate modeling language

Why are Contracts Useful? • • Precision in specification & design Separation of concerns Documentation Checking/Testing – dynamic (run-time) – static (compile-time) • Responsibility, enforceability, liability, …

Why Now? • Specifications are (still) a good idea! – focus shifted to critical properties rather than full correctness • • • Bug economics Test automation wall Moore’s law – abundant computational resources • Advances in research and technology – – model checking program analysis theorem proving analysis infrastructures

Overview • SLAM analysis engine – Static Driver Verifier • Other contract-checking tools – Vault (type checking) – ESC/Java (theorem proving) – ESP (dataflow analysis)

Rules Static Driver Verifier Read for understanding New API rules Development Precise API Usage Rules (SLIC) Defects Drive testing tools Software Model Checking 100% path coverage Source Code Testing

SLAM – Software Model Checking • SLAM innovations – boolean programs: a new model for software – model creation (c 2 bp) – model checking (bebop) – model refinement (newton) • SLAM toolkit – built on MSR program analysis infrastructure

SLIC • Finite state language for stating rules – monitors behavior of C code – temporal safety properties (security automata) – familiar C syntax • Suitable for expressing control-dominated properties – e. g. proper sequence of events – can encode data values inside state

State Machine for Locking state { enum {Locked, Unlocked} s = Unlocked; } Rel Unlocked Acq Rel Locked Acq Error Locking Rule in SLIC Ke. Acquire. Spin. Lock. entry { if (s==Locked) abort; else s = Locked; } Ke. Release. Spin. Lock. entry { if (s==Unlocked) abort; else s = Unlocked; }

The SLAM Process boolean program #include <ntddk. h> reachability check error path C 2 BP + Harness Bebop predicate abstraction SLIC Rule refinement predicates Newton feasibility check

Example Does this code obey the locking rule? do { Ke. Acquire. Spin. Lock(); n. Packets. Old = n. Packets; if(request){ request = request->Next; Ke. Release. Spin. Lock(); n. Packets++; } } while (n. Packets != n. Packets. Old); Ke. Release. Spin. Lock();

Example do { Ke. Acquire. Spin. Lock(); U L L if(*){ L Ke. Release. Spin. Lock(); U L U U E } } while (*); Ke. Release. Spin. Lock(); Model checking boolean program (bebop)

Example Is error path feasible in C program? (newton) do { Ke. Acquire. Spin. Lock(); U L n. Packets. Old = n. Packets; L L U L U U E if(request){ request = request->Next; Ke. Release. Spin. Lock(); n. Packets++; } } while (n. Packets != n. Packets. Old); Ke. Release. Spin. Lock();

Example Add new predicate b : (n. Packets. Old == n. Packets) to boolean program (c 2 bp) do { Ke. Acquire. Spin. Lock(); U L n. Packets. Old = n. Packets; b = true; L L U L U U E if(request){ request = request->Next; Ke. Release. Spin. Lock(); n. Packets++; b = b ? false : *; } } while (n. Packets != n. Packets. Old); !b Ke. Release. Spin. Lock();

Example b : (n. Packets. Old == n. Packets) do { Ke. Acquire. Spin. Lock(); U L b = true; b L if(*){ b L b U b L !b U b L U b U E Ke. Release. Spin. Lock(); b = b ? false : *; } } while ( !b ); Ke. Release. Spin. Lock(); Model checking refined boolean program (bebop)

Example b : (n. Packets. Old == n. Packets) do { Ke. Acquire. Spin. Lock(); U L b = true; b L if(*){ b L b U !b U Ke. Release. Spin. Lock(); b = b ? false : *; } } while ( !b ); Ke. Release. Spin. Lock(); Model checking refined boolean program (bebop)

Demo

SLAM Status • 2000 -2001 – foundations, algorithms, prototyping – papers in CAV, PLDI, POPL, SPIN, TACAS • March 2002 – Bill Gates review • May 2002 – Windows committed to hire two Ph. D. s in model checking to support Static Driver Verifier • July 2002 – running SLAM on 100+ drivers, 20+ properties • September 3, 2002 – made initial release of SDV to Windows (friends and family) • April 1, 2003 – made wide release of SDV to Windows (any internal driver developer) • September, 2003 – team of six in Windows working on SDV – researchers moving into “consultant” role • November, 2003 – demonstration at Driver Developer Conference

SLAM Results • Boolean program model has proved itself • Successful for device driver contracts – control-dominated safety properties – few boolean variables needed to do proof or find real errors • Counterexample-driven refinement – terminates in practice – incompleteness of theorem prover not an issue

Other Ways to Check Contracts • Type systems – Vault programming language – type system extended to allow simple pre/post • Theorem proving – ESC/Java checker – uses JML specification language (rich pre/post conditions) • Dataflow analysis – ESP – uses SLIC-like state machine language

Conclusions • The technology now exists for enforcing simple API contracts using static analysis • Rollout/adoption – first as out-of-band tools (i. e. , SLAM, ESP, Fugue) – next as in-band tools (part of language/compiler)

Thanks To Software Productivity Tools group members – Sriram Rajamani (SLAM) – Rob De. Line, Manuel Fahndrich (Vault/Fugue) SLAM summer interns – – Sagar Chaki, Todd Millstein, Rupak Majumdar (2000) Satyaki Das, Wes Weimer, Robby (2001) Jakob Lichtenberg, Mayur Naik (2002) Jakob Lichtenberg, Shuvendu Lahiri, Georg Weissenbacher, Fei Xie (2003) SLAM Visitors – Giorgio Delzanno, Andreas Podelski, Stefan Schwoon Static Driver Verifier: Windows Partners – Byron Cook, John Henry, Vladimir Levin, Con Mc. Garvey, Bohus Ondrusek, Abdullah Ustuner