272 Software Engineering Fall 2008 Instructor Tevfik Bultan
272: Software Engineering Fall 2008 Instructor: Tevfik Bultan Lectures 13: Modularity, Interfaces and Verification
Joint Work with My Students • Action Language Verifier (ALV) – Tuba Yavuz-Kahveci, University of Florida, Gainesville • Web Service Analysis Tool (WSAT) – Xiang Fu, Hofstra University • (co-advised with Jianwen Su) • Design for verification – Aysu Betin-Can, Middle East Technical University • Interface Grammars – Graham Hughes, Ph. D candidate http: //www. cs. ucsb. edu/~bultan/vlab/publications. html
Outline • Motivation • PART 1: Verification of Synchronization Policies in Concurrent Programs via Interfaces • PART 2: Verification of Conversations among Web Services via Interfaces • PART 3: Modular Verification with Interface Grammars • Conclusions
Model Checking Software • Model checking – An automated software verification technique – Exhaustive exploration of the state space of a program to find bugs • Systematically explore all possible behaviors of a program – look for violations of the properties of interest • assertion violations, deadlock • Software model checkers: Verisoft, Java Path. Finder (JPF), SLAM, BLAST, CBMC
Two Challenges in Model Checking • State space explosion – Exponential increase in the state space with increasing number of variables and threads • State space includes everything: threads, variables, control stack, heap • Environment generation – Finding models for parts of software that are • either not available for analysis, or • are outside the scope of the model checker
Modular Verification • Modularity is key to scalability of any verification technique – Moreover, it can help in isolating the behavior you wish to focus on, removing the parts that are beyond the scope of your verification technique • Modularity is also a key concept for successful software design – The question is finding effective ways of exploiting the modularity in software during verification
Interfaces for Modularity • How do we do modular verification? – Divide the software to a set of modules – Check each module in isolation • How do we isolate a module during verification/testing? – Provide stubs representing other modules (environment) • How do we get the stubs representing other modules? – Write interfaces • Interfaces specify the behavior of a module from the viewpoint of other modules • Generate stubs from the interfaces
Interfaces and Modularity: Basic Idea 1. Write interface specifications for the modules 2. Automatically generate stubs from the interface specifications 3. Automatically generated stubs provide the environment during modular verification
Three Applications I will talk about three different instantiations of this basic idea: 1. Verification of synchronization policies in concurrent programs using finite state interfaces 2. Verification of conversations among web services using finite state interfaces 3. Verification of sequential interactions using interface grammars
PART 1 Concurrency Controller Pattern for Synchronization
An Infinite State Model Checker Action Language Specification + CTL property Action Language Parser Action Language Verifier (ALV) Composite Symbolic Library Model Checker Counter-example Verified Not sure Omega Library Presburger Arithmetic Manipulator CUDD Package BDD Manipulator MONA Automata Manipulator
What Can One Do with ALV? • Check if a specification satisfies a CTL property • For example, check a Read-Write lock implementation integer nr; boolean busy; initial: !busy and nr=0; r_enter: r_exit: w_enter: w_exit: [!busy] nr : = nr+1; nr : = nr-1; [!busy && nr=0] busy : = true; busy : = false;
Read-Write Lock in Action Language module main() integer nr; boolean busy; restrict: nr>=0; initial: nr=0 and !busy; S : Cartesian product of variable domains defines the set of states module Reader. Writer() enumerated state {idle, reading, writing}; initial: state=idle; I : Predicates defining the initial states R : Atomic actions of a single process r_enter: state=idle and !busy and nr’=nr+1 and state’=reading; r_exit: state=reading and nr’=nr-1 and state’=idle; w_enter: state=idle and !busy and nr=0 busy’ and state’=writing; w_exit: state=writing and !busy’ and state’=idle; Reader. Writer: r_enter | r_exit | w_enter | w_exit; endmodule main: Reader. Writer*(); spec: invariant(busy => nr=0) spec: invariant(busy => eventually(!busy)) endmodule R : Transition relation of a process, defined as asynchronous composition of its atomic actions R : Transition relation of main, defined as asynchronous composition of finite but arbitrary number of reader-writer modules
Arbitrary Number of Threads? • How do we check arbitrary number of threads? • Counting abstraction – Create an integer variable for each thread state – Each variable counts the number of threads in a particular state – Generate updates and guards for these variables based on the specification • Counting abstraction is automated
Parameterized Read-Write Lock module main() integer nr; boolean busy; parameterized integer num. Reader. Writer; restrict: nr>=0 and num. Reader. Writer>=1; initial: nr=0 and !busy; module Reader. Writer() integer idle, reading, writing; initial: idle=num. Reader. Writer; r_enter: idle>0 and !busy and nr’=nr+1 and idle’=idle-1 and reading’=reading+1; r_exit: reading>0 and nr’=nr-1 and reading’=reading-1 and idle’=idle+1; w_enter: idle>0 and !busy and nr=0 and busy’ and idle’=idle-1 and writing’=writing+1; w_exit: writing>0 and !busy’ and writing’=writing-1 and idle’=idle+1 Reader. Writer: r_enter | r_exit | w_enter | w_exit; endmodule main: Reader. Writer(); spec: invariant(busy => nr=0) spec: invariant(busy => eventually(!busy)) endmodule
Read-Write Lock Verification with ALV Integers Booleans Cons. Time (secs. ) Ver. Time (secs. ) Memory (Mbytes) RW-4 1 5 0. 04 0. 01 6. 6 RW-8 1 9 0. 08 0. 01 7 RW-16 1 17 0. 19 0. 02 8 RW-32 1 33 0. 53 0. 03 10. 8 RW-64 1 65 1. 71 0. 06 20. 6 RW-P 7 1 0. 05 0. 01 9. 1
Read-Write Lock in Java class Read. Write. Lock { private Object lock. Obj; private int total. Read. Locks. Given; private boolean write. Lock. Issued; private int threads. Waiting. For. Write. Lock; public Read. Write. Lock() { lock. Obj = new Object(); write. Lock. Issued = false; } public void get. Read. Lock() { synchronized (lock. Obj) { while ((write. Lock. Issued) || (threads. Waiting. For. Write. Lock != 0)) { try { lock. Obj. wait(); } catch (Interrupted. Exception e) { } Action } total. Read. Locks. Given++; Language } } Verifier public void get. Write. Lock() { synchronized (lock. Obj) { threads. Waiting. For. Write. Lock++; How do we translate this to Action Language? } while ((total. Read. Locks. Given != 0) || (write. Lock. Issued)) { try { Verification of lock. Obj. wait(); } catch (Interrupted. Exception e) { Synchronization // } Java } threads. Waiting. For. Write. Lock--; Programs write. Lock. Issued = true; in
A Design for Verification Approach Our design for verification approach is based on the following principles: 1. Use of design patterns that facilitate automated verification 2. Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification 3. An assume-guarantee style modular verification strategy that separates verification of the behavior from the verification of the conformance to the interface specifications 4. A general model checking technique for interface verification 5. Domain specific and specialized verification techniques for behavior verification
Concurrency Controller Pattern Thread. A Shared Controller Thread. B Shared. Stub +a() +b() Helper classes Action +blocking() +nonblocking() -Guarded. Execute used at runtime used during interface verification used both times Controller -var 1 -var 2 +action 1() +action 2() int Controller. State. Machine +action 1() +action 2() Guarded. Command State. Machine Guarded. Command +guard() +update()
Concurrency Controller Pattern • Avoids usage of error-prone Java synchronization primitives: synchronize, wait, notify • Separates controller behavior from the threads that use the controller – Supports a modular verification approach that exploits this modularity for scalable verification
Reader-Writer Controller This helper class is provided. No need to rewrite it! class Action{ class RWController implements protected final Object owner; … RWInterface{ private boolean Guarded. Execute(){ int n. R; boolean busy; boolean result=false; final Action act_r_enter, act_r_exit; for(int i=0; i<gc. V. size(); i++) try{ final Action act_w_enter, act_w_exit; if(((Guarded. Command)gc. V. get(i)). guard()){ RWController() { ((Guarded. Command)gc. V. get(i)). update(); . . . result=true; break; } gcs = new Vector(); }catch(Exception e){} gcs. add(new Guarded. Command() { return result; public boolean guard(){ } return (n. R == 0 && !busy); } public void blocking(){ public void update(){busy = true; }} synchronized(owner) { ); while(!Guarded. Execute()) { act_w_enter = new Action(this, gcs); try{owner. wait(); } } catch (Exception e){} } public void w_enter(){ owner. notify. All(); } act_w_enter. blocking(); } } public boolean w_exit(){ public boolean nonblocking(){ return act_w_exit. nonblocking(); } synchronized(owner) { public void r_enter(){ boolean result=Guarded. Execute(); act_r_enter. blocking(); } if (result) owner. notify. All(); public boolean r_exit(){ return result; } return act_r_exit. nonblocking(); } }
Controller Interfaces • A controller interface defines the acceptable call sequences for the threads that use the controller • Interfaces are specified using finite state machines public class RWState. Machine implements RWInterface{ r_enter reading State. Table state. Table; final static int idle=0, reading=1, writing=2; public RWState. Machine(){. . . r_exit state. Table. insert("w_enter", idle, writing); idle } w_exit public void w_enter(){ state. Table. transition("w_enter"); writing } w_enter. . . }
Modular Design / Modular Verification Thread Modular Interface Verification Thread 1 Thread 2 Thread n Thread 2 Thread 1 Concurrent Program Interface Machine Interface Controller Shared Data Controller Behavior Modular Behavior Verification
Behavior Verification • Analyzing properties (specified in CTL) of the synchronization policy encapsulated with a concurrency controller and its interface – Verify the controller properties assuming that the user threads adhere to the controller interface • Behavior verification with Action Language Verifier – We wrote a translator which translates controller classes to Action Language – Using counting abstraction we can check concurrency controller classes for arbitrary number of threads
Interface Verification • A thread is correct with respect to an interface if all the call sequences generated by the thread can also be generated by the interface machine – Checks if all the threads invoke controller methods in the order specified in the interfaces – Checks if the threads access shared data only at the correct interface states
Interface Verification • Interface verification with Java Path. Finder – Verify Java implementations of threads – Correctness criteria are specified as assertions • Look for assertion violations • Assertions are in the State. Machine and Shared. Stub – Performance improvement with thread Isolation • thread modular verification
Thread Isolation: Part 1 • Interaction among threads • Threads can interact with each other in only two ways: – invoking controller actions – Invoking shared data methods • To isolate threads – Replace concurrency controllers with controller interface state machines – Replace shared data with shared stubs
Thread Isolation: Part 2 • Interaction among a thread and its environment • Modeling thread’s calls to its environment with stubs – File I/O, updating GUI components, socket operations, RMI call to another program • Replace with pre-written or generated stubs • Modeling the environment’s influence on threads with drivers – Thread initialization, RMI events, GUI events • Enclose with drivers that generate all possible events that influence controller access
Verification Framework Controller Behavior Machine Controller Classes Behavior Verification Counting Abstraction Action Language Verifier Concurrent Program Controller Interface Machine Thread Classes Thread Isolation Thread Class Interface Verification Java Path Finder
A Case Study: TSAFE Tactical Separation Assisted Flight Environment (TSAFE) functionality: 1. Display aircraft position 2. Display aircraft planned route 3. Display aircraft future projected route trajectory 4. Show conformance problems between planned and projected route
TSAFE Architecture User Radar feed <<TCP/IP>> Feed Parser Server Client Flight Database Event. Thread <<RMI>> Graphical Client Computation 21, 057 lines of code with 87 classes Timer
Reengineering TSAFE • Found all the synchronization statements in the code (synchronize, wait, notify. All) • Identified 6 shared objects protected by these synchronization statements • Used 2 instances of a reader-writer controller and 3 instances of a mutex controller for synchronization • In the reengineered TSAFE code the synchronization statements appear only in the Action helper class provided by the concurrency controller pattern
Behavior Verification Performance Controller Time(sec) Memory (MB) P-Time (sec) P-Memory (MB) RW 0. 17 1. 03 8. 10 12. 05 Mutex 0. 01 0. 23 0. 98 0. 03 Barrier 0. 01 0. 64 0. 01 0. 50 BB-RW 0. 13 6. 76 0. 63 10. 80 BB-Mutex 0. 63 1. 99 2. 05 6. 47 P denotes parameterized verification for arbitrary number of threads
Interface Verification Performance Thread Time (sec) Memory (MB) TServer-Main 67. 72 17. 08 TServer-RMI 91. 79 20. 31 TServer-Event 6. 57 10. 95 TServer-Feed 123. 12 83. 49 TClient-Main 2. 00 2. 32 TClient-RMI 17. 06 40. 96 TClient-Event 663. 21 33. 09
Effectiveness in Finding Faults • Created 40 faulty versions of TSAFE by fault seeding • Each version had at most one interface fault and at most one behavior fault – 14 behavior and 26 interface faults • Among 14 behavior faults ALV identified 12 of them – 2 uncaught faults were spurious • Among 26 interface faults JPF identified 21 of them – 2 of the uncaught faults were spurious – 3 of the uncaught faults were real faults that were not caught by JPF
Falsification Performance Thread Time (sec) Memory (MB) TServer-RMI 29. 43 24. 74 TServer-Event 6. 88 9. 56 TServer-Feed 18. 51 94. 72 TClient-RMI 10. 12 42. 64 TClient-Event 15. 63 12. 20 Concurrency Controller RW-8 Time (sec) 0. 34 Memory (MB) 3. 26 RW-16 1. 61 10. 04 RW-P 1. 51 5. 03 Mutex-8 0. 02 0. 19 Mutex-16 0. 04 0. 54 Mutex-p 0. 12 0. 70
Conclusions • ALV performance – Cost of parameterized verification was somewhere between concrete instances with 8 and 16 threads – Falsification performance was better than verification • Completeness of the controller properties – Effectiveness of behavior verification by ALV critically depends on the completeness of the specified properties • Concrete vs. parameterized behavior verification – When no faults are found, the result obtained with parameterized verification is stronger – However for falsification we observed that concrete instances were as effective as parameterized instances
Conclusions • JPF performance – Typically falsification performance is better than verification performance – In some cases faults caused execution of new code causing the falsification performance to be worse than verification performance • Thread isolation – Automatic environment generation for threads result in too much non-determinism and JPF runs out of memory – Dependency analysis was crucial for mitigating this • Deep faults were difficult to catch using JPF – Three uncaught faults were created to test this
Conclusions • Unknown shared objects – The presented approach does not handle this problem – Using escape analysis may help • We could not find a scalable and precise escape analysis tool • Environment generation – This is the crucial problem in scalability of the interface verification – Using a design for verification approach for environment generation may help
PART 2 Peer Controller Pattern for Web Services
Web Services WSCDL Composition WSBPEL Service WSDL Message SOAP Type XML Schema Data XML Web Service Standards Implementation Platforms Interaction Microsoft. Net, Sun J 2 EE • Loosely coupled, interaction through standardized interfaces • Standardized data transmission via XML • Asynchronous messaging • Platform independent (. NET, J 2 EE)
Web Service Conversations • A composite web service consists of – a finite set of peers – and a finite set of message classes • The messages among the peers are exchanged using reliable and asynchronous messaging – FIFO and unbounded message queues • A conversation is a sequence of messages generated by the peers during an execution
Properties of Conversations • The notion of conversation enables us to reason about temporal properties of the composite web services • LTL framework extends naturally to conversations – LTL temporal operators X (ne. Xt), U (Until), G (Globally), F (Future) – Atomic properties Predicates on message classes (or contents) Example: G ( payment F receipt ) • Model checking problem: Given an LTL property, does the conversation set satisfy the property?
Top-Down vs. Bottom-Up confirm query Agent Customer suggest Conversation Protocol Agent reserve ? reserve G(query F(confirm)) confirm query suggest Customer ? query Hotel !query ? reserve ? suggest !suggest Input Queue !confirm !reserve ? suggest !suggest Conversation ? confirm . . . ? G(query F(confirm))
Realizability Analysis • Conversation protocol specifies the global communication behavior – How do we implement the peers? • How do we obtain the contracts that peers have to obey from the global contract specified by the conversation protocol? • Project the global protocol to each peer – By dropping unrelated messages for each peer • A conversation protocol is realizable if the following are equal: – Conversations specified by the conversation protocol – Conversations generated by the projected services
Synchronizability Analysis • A composite web service is synchronizable, if its conversation set does not change – when asynchronous communication is replaced with synchronous communication • If a composite web service is synchronizable we can check its properties about its conversations using synchronous communication semantics – For finite state peers this is a finite state model checking problem
Web Service Analysis Tool (WSAT) Web Services BPEL (bottom-up) Front End BPEL to GFSA Analysis Back End Intermediate Representation Guarded automata Synchronizability Analysis skip Conversation Protocol (top-down) GFSA parser Guarded automaton s GFSA to Promela ces c (synchronous u s communication) fai l GFSA to Promela (bounded queue) Realizability Analysis fail success GFSA to Promela (single process, no communication) Verification Languages Promela
Checking Service Implementations There are some problems: • People write web service implementations using programming languages such as Java, C#, etc. • Synchronizability analysis works on state machine models • How do we generate the state machines from the Java code? Synchronizability Analysis Checking Service Implementations
Design for Verification Approach Use the same principles: 1. Use of design patterns that facilitate automated verification 2. Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification 3. An assume-guarantee style modular verification strategy that separates verification of the behavior from the verification of the conformance to the interface specifications 4. A general model checking technique for interface verification 5. Domain specific and specialized verification techniques for behavior verification
Peer Controller Pattern Application. Thread Communicator Peer. Servlet Communication. Interface Communication. Controller State. Machine session. Id Thread. Container used at runtime used during interface verification used both times Red Bordered classes are the ones the user has to implement
Peer Controller Pattern • Eases the development of web services • Uses Java API for XML messaging (JAXM) – Asynchronous communication among peers • Supported by a modular verification technique – Behavior Verification: Checks properties of conversations of a web service composed of multiple peers • assuming that peers behave according to their interfaces – Interface Verification: Checks if each peer behaves according to its interface
Modular Design / Modular Verification Peer Modular Interface Verification Peer 1 Peer 2 Peer n interface Peer 2 interface Peer 1 Composite Service Interface Machine Conversation Behavior Modular Conversation Verification
Verification Framework Thread Peer Thread State Machines Composite Service WSAT Promela Translation Synchronizability Analysis Peer State Machine Peer Code Promela Conversation Verification Interface Verification Spin Java Path Finder
Behavior Verification • Uses WSAT for synchronizability analysis • Uses Spin model checker for conversation verification – Automated translation to Promela using WSAT • Spin is a finite state model checker – We have to bound the channel sizes, session numbers, message types • Synchronizability analysis – Enables us to verify web services efficiently by replacing communication channels with channels of size 0 (i. e. , synchronous communication) – The verification results hold for unbounded channels
Interface Verification • If the call sequence to the Communicator class is accepted by the state machine specifying the interface, then the peer implementation conforms to the behavior in the contract • Uses JPF model checker
Interface Verification • Isolated check of individual peer implementations – Communication. Controller is replaced with Communicator. Interface – Drivers simulating other peers are automatically generated • State Space reduction – Usage of stubs – Each session is independent • just need to check each peer for one session
Examples • We used this approach to implement several simple web services – Travel agency – Loan approver – Product ordering • Performance of both interface and behavior verification were reasonable
Interface Verification with JPF for Loan Approver Threads T (sec) M (MB) Customer 8. 86 3. 84 Loan Approver 9. 65 4. 7 Risk Assesor 8. 15 3. 64
Behavior Verification • Sample Property: Whenever a request with a small amount is sent, eventually an approval message accepting the loan request will be sent. • Loan Approval system has 154 reachable states – because queue lengths never exceeds 1 • Behavior verification used <1 sec and 1. 49 MB • SPIN requires restricted domains – Have to bound the channel sizes bounded message queues • In general there is no guarantee these results will hold for other queue sizes – Using synchronizability analysis we use queues of size 0 and still guarantee that the verification results hold for unbounded queues!
Conclusions • We were able to use our design for verification approach based on design patterns and behavioral interfaces in different domains • Use of domain specific behavior verification techniques has been very effective • Interface verification is challenging • Model checking research resulted in various verification techniques and tools which can be customized for specific classes of software systems • Automated verification techniques can scale to realistic software systems using design for verification approach
Conclusions • Once the behavior is isolated (using concurrency controller or peer controller patterns) behavior verification was quite efficient • Interface verification was very hard • It is necessary to find effective behavioral interface specification and verification techniques
PART 3 Interface Grammars
Interface Grammars Interface Compiler Component B Stub Component A Model Checker Component A
An Example • An interface grammar for transactions – Specifies the appropriate ordering for method calls to a transaction manager Start → Base → | Tail → | Base begin Tail Base ε commit rollback – Method calls are the terminal symbols of the interface grammar
�An Example • Consider the call sequence begin rollback begin commit Start → Base → | Tail → | • Here is a derivation: Start Base begin Tail Base begin rollback begin commit Base begin Tail Base ε commit rollback
Another Example • This interface can also be specified as a Finite State Machine (FSM) begin commit rollback • However, the following grammar, which specifies nested transactions, cannot be specified as a FSM Start → Base → | Tail → | Base begin Base Tail Base ε commit rollback
Yet Another Example • Let’s add another method called setrollbackonly which forces all the pending transactions to finish with rollback instead of commit • We achieve this by extending the interface grammars with semantic predicates and semantic actions Start → Base → Tail | | → | «r: =false; l: =0» Base begin «l: =l+1» Base Tail «l: =l-1; if l=0 then r: =false» Base setrollbackonly «r: =true» Base ε «r=false» commit rollback
Interface Grammar Translation • Our interface compiler translates interface grammars to executable code: – the generated code is the stub for the component • The generated code is a parser that – parses the incoming method calls – while making sure that the incoming method calls conform to the interface grammar specification
Interface Grammar Verification with Interface Grammars parser stack Component Stub Interface Compiler parse table Top-down parser semantic predicates and semantic actions Program method invocation (lookahead) Model Checker
A Case Study • We wrote an interface grammar for the EJB 3. 0 Persistence API – This is an API specification for mapping Java object graphs to a relational database – Hibernate is an implementation of this API • Hibernate distribution contains several example test clients that are designed to fail and test exceptional behavior by violating the interface specification
A Case Study, Continued • We used these simple clients to check the fidelity of the stub generated from our interface specification – We used the JPF software model checker • None of these examples can run under JPF directly • Time taken to develop the interface was dominated by the need to understand EJB Persistence first – about a couple of hours
Experiments: Falsification Client 1 Client 2 Client 3 sec. # obj. # iter. 1. 9 1. 8 2. 1 1 5 1. 9 1. 8 2. 1 5 1 1. 9 1. 8 2. 1 5 5 Client 4
Experiments: Verification Client 1 Client 2 Client 3 sec. # obj. # iter. 3. 1 1. 9 1. 8 2. 5 1 1 26. 9 11. 1 9. 9 17. 1 1 5 10. 1 5. 9 4. 2 11. 8 5 1 126. 3 63. 9 43. 5 138. 5 5 5 Client 4
A Case Study, Continued • For these simple clients, interface violations can be detected by JPF in a couple of seconds using the EJB stub generated by our interface compiler – Falsification time does not increase with the number of operations executed or the number of objects created by the clients • When we fix the errors, JPF is able to verify the absence of interface violations – Verification time increases with the number of operations executed or the number of objects created by the clients
Interface Grammars: Uni/Bidirectional Interface • Interface grammars can be – Unidirectional: No callbacks Callee Caller Interface – Bidirectional: Need to handle Callbacks Comp B Comp A
Interface Grammars: Client/Server Interface • Interface grammars can be used for – Client verification: Generate a stub for the server – Server verification: Generate a driver for the server Stub Client Driver Server Interface Compiler
Semantic Elements in JML • To handle both client and server side verification – We need to generate stubs and drivers from the same specification • Semantic predicate in one direction becomes a semantic action in the other direction and visa versa • We focused on a subset of Java Modeling Language – A restricted subset that is reversable – Semantic predicates and actions are written in this subset – Interface compiler automatically generates code from them both for client and server side verification
Interface Grammars and Data • A crucial part of the interface specification is specifying the allowable values for the method arguments and generating allowable return values • Approach 1: These can be specified in the semantic actions and semantic predicates of the interface grammars • Approach 2: Can we specify the constraints about the arguments and return values using the grammar rules? – Yes, grammar productions can be used to specify the structure of most recursive data structures.
Checking Arguments • A crucial part of the interface specification is specifying the allowable values for the method arguments and generating allowable return values • In what I discussed so far all these are done in the semantic actions and semantic predicates • The question is, can we specify the constraints about the arguments and return values using the grammar rules – Recursive data structures are especially good candidates for this!
Shape Types • Shape types [Fradet, Metayer, POPL 97] provide a formalism for specifying recursive data structures • It is a specification formalism based on graph grammars • Shape types can be used to specify the connections among the heap allocated objects • Objects become the parameters of the nonterminals and the constraints on the connections among the objects are specified on the right-hand-sides of the grammar rules (similar to semantic predicates)
Shape Type for Doubly Linked List → → → Doubly Lx Lx p x, prev x null, L x next x y, prev y x, L y next x null p prev next 1 next 2 prev next 3 4 prev Doubly p 1, prev 1 null, L 1 next 1 2, prev 2 1, L 2 next 2 3, prev 3 2, L 3 next 3 4, prev 4 3, L 4 next 4 null next
Shape Type for Binary Tree Bintree Bx Bx → → → p x, B x left x y, right x z, B y, B z left x null, right x null p left 2 right 1 right left 4 right left 5 right 3 right
Extension to Interface Grammars • In order to support shape types we extend the interface grammars as follows: – We allow nonterminals with parameters • This extension is sufficient since the constraints about the connections among the objects can be stated using semantics predicates and semantic actions
Interface Grammars + Shape Types → → → Doubly Lx Lx p x, prev x null, L x next x y, prev y x, L y next x null Doubly[x] → ensure x == new(Node) && x. get. Prev() == null; L[x] → Node y; ensure y == new(Node) && x. get. Next() == y && y. get. Prev() == x; L[y] | ensure x. get. Next() == null;
Interface Grammars + Shape Types → → → Bintree Bx Bx p x, B x left x y, right x z, B y, B z left x null, right x null Bintree[x] → ensure x == new(Node); B[x] → Node y, z; ensure y == new(Node) && z == new(Node) && x. get. Left() == y && x. get. Right() == z ; B[y] B[z] | ensure x. get. Left() == null && x. get. Right() == null;
Objection Generation vs. Validation • The use of shape types in interface grammars has two purposes – For the objects that are passed as method arguments we need to check that their shape is allowed by the shape type • We call this object validation – For the objects that are returned by the component we need to generate an object that is allowed by the shape type • We call this object generation
Object Generation vs. Validation • Object generation and validation tasks are broadly symmetric – The set of nonterminals and productions used for object generation and validation are the same and are dictated by the shape type specification – In object generation semantic actions are used to set the fields of objects to appropriate values dictated by the shape type specification – In object validation these are constraints that are checked using semantic predicates specified as guards • Given the semantic elements specified in JML, our interface compile generates code both for object generation and validation
Object Generation vs. Validation • There is a minor problem with object validation • In shape type specifications, the assumption is that there is no aliasing among the objects unless it is explicitly specified • This assumption is easy to enforce during object generation since every new statement creates a new object that has nothing else pointing to it • In order to enforce the same constraint during object validation we need to make sure that there is no unspecified aliasing – This can be enforced by using a hash-set for storing and propagating all the observed objects
Modular Verification of Web Services • We applied our modular verification approach based on interface grammars to both client and server side verification of Web services
Interface Grammars and Web Services Our approach: 1. A WSDL-to-interface grammar translator automatically generates grammar productions that generate and/or validate XML arguments and return values 2. User adds control flow constraints by modifying the grammar 3. Interface compiler automatically generates a stub for client side verification and a driver for server-side verification
Interface Grammars for Web Services
Another Case Study: AWS-ECS • We tested the Amazon E-Commerce Web Service (AWSECS) using our approach • AWS-ECS WSDL specification lists 40 operations – that provide differing ways of searching Amazon’s product database • We focus on the core operations: – Item. Search, Cart. Create, Cart. Add, Cart. Modify, Cart. Get, Cart. Clear
Client-side Verification • For client verification we used a demonstration client provided by Amazon • This client does not check any constraints such as – You should not try to insert an item to a shopping cart before creating a shopping cart • When such requests are sent to AWS-ECS they would return an error message • Using our approach we can easily check if the client allows such erroneous requests • Falsification time changes with the type of faults we are looking for (data or control errors), changes from 10 to 60 seconds
AWS-ECS: Server Verification • Our interface compiler automatically generates a driver that sends sequences of requests to AWS-ECS server and checks that the return values conform to the interface specification • The driver is a sentence generator – It generates sequences of SOAP requests based on the interface specification • We used two algorithms for sentence generation: – A random sentence generation algorithm – Purdom’s algorithm: A directed sentence generation algorithm that guarantees production coverage
Directed Sentence Generation • Number of sentences generated: 5 • Average derivation length: 24 • Average number of SOAP requests/responses: 3. 8 • Verification time: 20 seconds
Random Sentence Algorithm • Number of sentences generated: 100 • Average derivation length: 17. 5 • Average number of SOAP requests/responses: 3. 2
Server-side verification • We found two errors during server side verification – Errors were discovered within 10 seconds • These errors indicate a mismatch between the interface specification and the server implementation • It may mean that we misunderstood the description of the Web service interface • It may also mean that there is an error in the service implementation
Conclusions • Modular verification is a necessity • Interfaces are crucial for modular verification • Interface grammars provide a new specification mechanism for interfaces • Interface grammars can be used for automated stub and driver generation leading to modular verification
Conclusions • Behavioral interfaces can be useful both – From software design perspective by enabling better modular designs – From verification perspective by enabling more efficient modular verification • The challenge is to find behavioral interface specification mechanisms that serve both of these goals
Related Work: Modular Verification • Clarke, Long, Mc. Millan, Compositional Model Checking • Henzinger, Qadeer, Rajamani, Assume Guarantee Reasoning in Model Checking • Flanagan, Qadeer, Thread-Modular Model Checking • Krishnamurthi, Fisler, Modular Verification of Feature Oriented Programs
Related Work: Design for Verification • Meyer, Design by Contract • Flanagan, Leino, et al. ESC Java • Mehlitz, Penix, Design for Verification Using Design Patterns • Sarna-Starosta, Stirewalt, Dillon, Design for Verification for Synchronization
Related Work: Interfaces • L. de Alfaro and T. A. Henzinger. Interface automata. • O. Tkachuk, M. B. Dwyer, and C. Pasareanu. Automated environment generation for software model checking. • T. Ball and S. K. Rajamani. SLAM interface specification language. • G. T. Leavens et al. : JML
Related: Grammar-based Testing • A. G. Duncan, J. S. Hurchinson: Using attributed grammars to test designs and implementations • P. M. Maurer: Generating test data with enhanced context free grammars • P. M. Maurer: The design and implementation of a grammar-based data generator • E. G. Sirer and B. N. Bershad: Using production grammars in software testing
- Slides: 103