Apply Program Model Checking Qian Ling Oct 28

Apply Program Model Checking Qian Ling Oct. 28, 2004 Bell Labs Research China

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool Verisoft Tool SLAM Tool • Use Spin tool to check TTS Server • Future Work 2

Motivation • Testing of concurrent programs – VMS TTS Server Program – OS Adapter Project – POSIX thread impl. on Vx. Works 5. 4 • Progress in Program Model Checking – New tech and tools developed 3

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool Verisoft Tool SLAM Tool • Use Spin tool to check TTS Server • Future Work 4

SW Quality Control Tech • Software Testing – – Unit/integration/system test Functional test, Conformance test Stress test API test, etc • Code Review – Peer Review – Design Review • Program Model Checking – Formal Method – Software Verification • Software Quality Assurance 5

Software Testing “Sequential, Stand-alone” 6

Code Review “Static” 7

Facing Concurrent & Distributed Software It is not too surprising that the behavior of even non-buggy distributed applications can easily defy our human reasoning skills. Gerard J. Holzmann 8

Facing Concurrent & Distributed Software, • • • cont Deadlock Livelock, starvation Underspecification – unexpected reception of messages • Overspecification – Dead code • Violations of constraints – Buffer overruns – Array bounds violations • Assumptions about speed – Logical correctness vs. real-time performance Really Hard Issue 9

What is Model Checking? • [Clarke & Emerson 1981] “Model checking is an automated technique that, given a finite-state model of a system and a logical property, systematically checks whether this property holds for (a given initial state in) that model. ” • Model checking tools automatically verify whether M |= φ Holds where M is a (finite-state) model of a system and property φ is stated in some formal notation • Problem: state space explosion • SPIN is one of the most powerful model checker 10

Classic v. s. Modern MC Waterfall SW Lifecycle Process 11

Classic Model Checking 12

Modern Model Checking • Abstraction: To fight with explosion • Focus on Modern Model Checking 13

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool Verisoft Tool SLAM Tool • Use Spin tool to check TTS Server • Future Work 14

Tools for program model Checking • SPIN – Gerard J. Holzmann (Bell-Labs), 1991~ – Proving the correctness of process interaction, asynchronized control • Verisoft – Patrice Godefroid (Bell Labs), 1996~ – Runtime analyses for multi-process programs • SLAM – Thomas Ball (MS Research), 1999~ – Static analyses for “API Usage Rules” in programs 15

Tools for program model Checking, cont • Eraser – Stefan Savage (University of Washington) & DEC, 1997~ – Dynamic detecting data-race in lock-based multithread programs • FLAVERS – Matthew B. Dwyer, L. A. Clarke (University of Kansas State/Massachusetts Amherst) &Do. D, DAAH, NSF, IBM… , 1999~ – Check if all executions adhere to specification, data flow analysis, stack • SMV – CMU, 1992~ – Symbolic model checker • … 16

Tool Comparison • • Modeling Languages Static Analyze v. s. Dynamic Explicit State space? Scope of verification – Safety only – Acceptation cycle, etc • Stages in development – Design phase – Code Ready – Incremental development 17

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool – The Verisoft Tool – The SLAM Tool • Use Spin tool to check TTS Server • Future Work 18

The SPIN Model Checker • SPIN = Simple Promela Interpreter – is a tool for analysing the logical consistency of concurrent systems, specifically of data communication protocols. – Concurrent systems are described in the modelling language called Promela. – state-of-the-art model checker 19

Promela • Promela = Protocol/Process Meta Language – allows for the dynamic creation of concurrent processes. – communication via message channels can be defined to be • synchronous (i. e. rendezvous), or • asynchronous (i. e. buffered). – resembles the programming language C – specification language to model finite-state systems 20

Modern Model Checking with SPIN 21

Foundation of SPIN • Based on automata theoretic model – System Model • • • Vardi & Wolper 83 Translate process tmpl to automaton State space=product of all automata Global behavior modeled by an automaton – Specification • Correctness requirement in LTL formula • Convert LTL to Büchi automaton – Verification • global automaton× Büchi automaton – (Accepted language = empty) not satisfied – (Accepted language ‡ empty) satisfied behaviors • SPIN formalize the erroneous behaviors – Prove such behavior are impossible – Provide detail matched behaviors 22

Algorithms of SPIN • Nested Depth-First Search (Cycles detection) – 1 st search: from initial state, find accepting states – 2 nd search (nested): from all accepting states • LTL to Büchi automaton – Normalize LTL formulas recursive conversion – Make use of Promela condition statement • Partial Order Reduction Problem coverage = M / (N*S) M=2^m: memory space in bytes – Represent classes of execution sequence with one single N=2^n: true # of Reachable states path, statically • Memory Management S=2^s: bytes for each state – State Compression (60%~80% reduction) • Separate local state (small) and global state • Global state use index to refer to local state – Bit-State Hashing (Improve problem coverage 1) • • • 2 bits are used to store a reachable state 2 bits: F 1(S 1) = M[m 1], F 2(S 2)=M[m 2] Collision problem 23

Debug with SPIN • Correctness Requirement: – Inline assertions • assert(…) – End-state label / deadlock • endxxx: – Progress-state label / livelock Commonly Used • progressxxx: – Acceptance-state label • acceptxxx: – LTL properties • Make use of LTL specification – Never-claims • never {…} 24

Extensions to SPIN (1) • d. Spin = dynamic SPIN – R. Iosif & R. Sisto, Kanas State University, 1999~ – Solve Limitations in SPIN, • Object creation & deletion • Pointer and reference variables • Polymorphic function calls – Solutions • Memory management – Dynamic memory allocation – Reference mechanism • Functional management – Function declaration, call and reference – Local scoping 25

Extensions to SPIN (2) • Fea. Ver/Modex = Model Extractor for C – Gerard J. Holzmann, Bell Labs, 1998~ – Extract Promela model from ANSI C – 3 phases: • Parsing: C Full Parse Tree • Interpretation: Tabled Abstraction • Optimization – Applications: • Lucent Path. Star™ Access Server 26

Extensions to SPIN (3) • JPF 1/JPF 2 = Java Path Finder – Automated Software Engineering Group, NASA – JPF 1, since 1998 • A translator from Java 1. 0 to Promela • Suitable for multi-threaded Java programs • Applications – Find deadlocks in Game Server – Analyze Remote Agent – JPF 2, since 2000 • • • Work on Java Byte Code, 100% Java support Garbage Collection Static analysis dependencies for partial-order reduction Contain runtime race-cond analysis by “Eraser” Algorithm Use Bandera to calculate slices 27

Extensions to SPIN (4) • Bandera Toolset – Santos Laboratory, since 2000 – source code FSM model checker, SPIN/SMV • Analyze, • Abstraction • Transformation – More than just a SPIN extension! 28

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool – The Verisoft Tool – The SLAM Tool • Use Spin tool to check TTS Server • Future Work 29

Veri. Soft • A tool for systematically testing concurrent reactive software • Stateless search, for: • Applications: – Deadlocks – 4 ESS Heart-Beat Monitor – Livelocks debugging and unit testing – Assertion violations – Wave. Star 40 G R 4 – Divergence integration and system testing – 7 R/E Packet. Star Gateway Solutions Feature Server unit testing – CDMA Call Processing 30 Library testing • Replay Error Scenarios

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool – The Verisoft Tool – The SLAM Tool • Use Spin tool to check TTS Server • Future Work 31

SLAM • Check Temporal safety property in sequential C programs • Based on Boolean Program • Solutions: – Preparation: • Model conterexample in SLIC • C BP (C 2 BP) – Verification: • • BEBOP: perform NEWTON: refine reachability analysis of BP BP by discover additional predicts – Error in BP Error in P • Applications – API in Windows XP device drivers 32

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool Verisoft Tool SLAM Tool • Use Spin tool to check TTS Server • Future Work 33

VMS TTS Server • Multi-threaded – 1 server thread – 1~10 tts working threads – 1~10 IBM viavoice synthesis threads • Asynchronzed Message Q – server working threads • Synchronized – synthesis threads working threads 34

VMS TTS Server, cont • A Sequence Diagram 35

Purpose • Check safety properties – potential deadlocks – Unexpected end states – Unreachable code • Check if it works for 10 V V instances – Currently we have only 1 instance • Check about the Msg Loss 36

Result in Simulation • No Error found in program (simulation) – 1 bug found in promela model • Deadlock in server thread • Forget to add lock when access globals in abstr. – There are un-reached code – Abstraction is not a easy work • • Programs works for 10 threads There are message loss – Under high stress • Every thread are full loaded – Increase # of buffers fewer loss – Increase # of instance fewer loss – Decrease synthesis delay • As for performance – Simulation is not trustable 37

Result in PAN 38

Restrictions, Lessons • No Timer in Promela – Use loop to simulate a timer – Different behavior in simulation & PAN • Hard to locate an error – Too many messages – Generally, subtle errors occur after quite a long time • Platform dependant APIs are hard to be abstracted – What is the size of the Win 32 thread message queue? 39

Agenda • • • Motivation Program Model Checking Tools – The Spin Tool Verisoft Tool SLAM Tool • Use Spin tool to check TTS Server • Future Work 40

Future Work • More applications – OS Adapter Project – POSIX thread –… • Tool enhancements – Locate bug easier ? – Timer ? • Automata LTL ? 41

Reference Site • SPIN – http: //spinroot. com/spin/whatispin. html • Verisoft – http: //cm. bell-labs. com/who/god/verisoft/ • SLAM – http: //research. microsoft. com/slam/ 42

Thanks! & Suggestions!