Efficient Modular Glass Box Software Model Checking Michael
Efficient Modular Glass Box Software Model Checking Michael Roberson Chandrasekhar Boyapati The University of Michigan
Software Model Checking Exhaustively test programs On all possible inputs On all possible schedules Up to finite bounds
Binary Tree State Space Initial State Space Explosion
State Space Reduction Many software model checkers Verisoft, JPF, CMC, SLAM, Blast, Magic, … Many state space reduction techniques Partial order reduction Predicate abstraction Effective for control-oriented properties Our work is on Glass Box Software Model Checking Effective for data-oriented properties Significantly more efficient than previous model checkers
Modular Glass Box Checking Check modules against abstractions Check program replacing modules with abstractions Modular glass box model checking is important Further improve scalability of glass box checking Modular glass box model checking is nontrivial
Modular Checking Modular checking in traditional model checkers Initial State of Module Check outputs at each step Initial State of Abstraction
Modular Glass Box Checking We can't reach this transition! We cannot use reachability through transitions Programmers must provide a class invariant State space includes all states that satisfy the invariant Programmers must provide an abstraction function We use it to generate the abstraction of each state
Modular Glass Box Checking Choose a state Run one operation Check outputs Generate abstraction on post-state c 2 c 1_output Abstraction Generate abstraction c 1 Abstraction Operation a 2' Check for equality Operation a 1 a 2 a 1_output Equal
Outline Motivation Example Approach Experimental Results Related Work Conclusions
Integer Counter class Integer. Counter { Map map = new Search. Tree(); int max_frequency = 0; int most_frequent = 0; new Abstract. Map(); void count(int i) { Integer frequency = (Integer)map. get(i); if (frequency == null) frequency = new Integer(0); map. insert(i, new Integer(frequency+1)); Frequencies are stored in a Map Modular Approach: Count an integer Replace Module with Abstraction if (frequency >= max_frequency) { max_frequency = frequency; most_frequent = i; } } int get_most_frequent() { return most_frequent; } int get_max_frequency() { return max_frequency; } } Return most frequent integer Return frequency of most frequent integer
Module vs Abstraction Search. Tree vs Implements Map Abstract. Map Implements Map get, insert, delete Balanced binary tree Efficient execution Linked list Simple execution Larger state space Smaller state space 5 2 1 6 4 1 2 4 5 6
Checking Search. Tree Choose a state Run one operation Check outputs Generate abstraction on post-state c 14 c 2 Abstraction Generate abstraction 1 6 Abstraction Operation 5 2 a 2' Check for equality Operation a 1 a 2 Equal
Checking Search. Tree Choose a state Run one operation Check outputs Generate abstraction on post-state c 2 4 Abstraction Generate abstraction 1 6 Abstraction Operation 5 2 a 2' Check for equality Operation a 1 1 2 4 5 6 a 2 Equal
Checking Search. Tree Choose a state Run one operation Check outputs Generate abstraction on post-state 1 6 5 2 1 4 c 24 6 3 Abstraction Generate abstraction 2 Abstraction insert(3, x) Operation 5 a 2' Check for equality insert(3, x) Operation 1 2 4 5 6 a 2 1 2 3 4 5 6 Equal
Checking Search. Tree Choose a state Run one operation Check outputs Generate abstraction on post-state 1 6 5 2 1 4 6 4 3 c 1_output Abstraction Generate abstraction 2 Abstraction insert(3, x) 5 a 2' Check for equality insert(3, x) 1 2 3 4 5 6 1 2 4 5 6 a 1_output Equal
Checking Search. Tree Choose a state Run one operation Check outputs Generate abstraction on post-state 1 6 5 2 1 4 6 4 3 Abstraction Generate abstraction 2 Abstraction insert(3, x) 5 a 2' 1 2 3 4 5 6 Check for equality insert(3, x) 1 2 4 5 6 1 2 3 4 5 6 Equal
Checking Search. Tree Choose a state Run one operation Check outputs Generate abstraction on post-state 1 6 5 2 1 4 6 4 3 Abstraction Generate abstraction 2 Abstraction insert(3, x) 5 1 2 3 4 5 6 Check for equality insert(3, x) 1 2 4 5 6 1 2 3 4 5 6 Equal
Glass Box Pruning An insert operation only touches one path Insert behaves similarly on many states insert(3, x) 5 5 2 1 insert(3, x) 5 2 6 1 4 2 4 1 3 5 6 4 7 4 5 2 6 2 1 5 2 4 3 6 4 3 7
Glass Box Pruning We don't need to check more than one of these We can prune all others from the state space insert(3, x) 5 5 2 1 1 4 2 7 4 5 6 4 6 PRUNED 5 3 5 2 6 1 insert(3, x) 2 1 5 2 4 3 6 4 3 7
Glass Box Pruning We check each tree path, not each tree This reduces the state space dramatically insert(3, x) 5 5 2 1 1 4 2 7 4 5 6 4 6 PRUNED 5 3 5 2 6 1 insert(3, x) 2 1 5 2 4 3 6 4 3 7
Checking Integer. Counter with Search. Tree vs Integer. Counter with Abstract. Map Integer. Counter' Integer. Counter 5 2 1 1 2 4 5 6 4 4 5 1 6 2 2 2 6 1 6 2 2 1 4 5 1 2 4 5 6 6 5 5 6 5 4 1 6 4 1 Smaller state space Better state space reduction Faster analysis
Outline Motivation Example Approach Program specification Search algorithm State space representation State space reduction Experimental Results Related Work Conclusions
Module Implementation class Search. Tree implements Map { class Node { int key; Object value; Node left; Node right; } Operation c 1 c 2 Node root; Object get(int key) { /*. . . */ } void insert(int key, Object value) { /*. . . */ } void remove(int key) { /*. . . */ } } Map interface methods
Abstraction class Abstract. Map implements Map { class Node { Object key; Object value; Node next; } Node head; Object get(int key) { /*. . . */ } void insert(int key, Object value) { /*. . . */ } void remove(int key) { /*. . . */ } @Declarative boolean equal. To(Abstract. Map m) { /*. . . */ } } Declarative methods Subset of Java Operation a 1 Free of side effectsa 2 a 2' Used for specification Aid our analyses a 2 methods Map interface Equality test
Module Specification c 1 Abstraction class Search. Tree implements Map { class Node { int key; Object value; Node left; Node right; } Node root; /*. . . Map operations. . . */ @Declarative boolean rep. Ok() { /*. . . */ } Abstract. Map abstraction() { /*. . . */ } } a 1 Module Invariant Abstraction function
Approach Program specification Search algorithm State space representation State space reduction
Search Algorithm Choose an unchecked valid state insert(3, x) 5 2 1 6 4 @Declarative boolean rep. Ok() { /*. . . */ }
Search Algorithm Generate its abstraction insert(3, x) 5 2 6 1 4 Abstract. Map abstraction() { /*. . . */ } 1 2 4 5 6
Search Algorithm Run the operation on both states insert(3, x) 5 5 2 2 6 1 1 4 6 4 3 void insert(int key, Object /*. . . */ } 1 2 4 5 6 1 value) { 2 3 4 5 6
Search Algorithm Generate the post-state abstraction insert(3, x) 5 5 2 2 6 1 1 4 2 4 5 6 4 3 Abstract. Map abstraction() { /*. . . */ } 1 6 1 2 3 4 5 6
Search Algorithm Check invariant and abstraction equality @Declarative boolean rep. Ok() { /*. . . */ } insert(3, x) 5 2 6 1 1 4 2 4 5 6 4 3 @Declarative boolean equal. To(Abstract. Map m) { /*. . . */ } 1 6 1 2 3 4 5 6
State Space Reduction Identify and prune similar states insert(3, x) 5 5 2 1 6 1 2 4 2 7 4 5 6 4 6 PRUNED 5 3 5 2 4 1 insert(3, x) 2 1 5 2 4 3 6 4 3 7
Search Algorithm Let S be the states that satisfy rep. Ok() While S is not empty Choose a state s in S. Check s. Let P be the set of states similar to s S=S-P Need efficient representation and operations for these sets!
Approach Program specification Search algorithm State space representation State space reduction
Representation Represent a set as a boolean formula Encode each field as bits (b 0, b 1, …) Constrain the bits using boolean operations n 1 key = {b 0, b 1} value = {b 2, b 3} left = {b 4} right = {b 5} n 1. left = null || n 1. key > n 2. key Øb 4 Ú (b 1 Ù Øb 7) Ú ((b 1 Ú Øb 7) Ù b 0 Ù Øb 6) n 2 key = {b 6, b 7} value = {b 8, b 9} left = {} right = {} . . .
Representation Initialize to set of states that satisfy invariant Construct a formula describing invariant @Declarative boolean rep. Ok() { /*. . . */ } boolean formula
Representation Initialize to set of states that satisfy invariant Construct a formula describing invariant @Declarative boolean rep. Ok() { /*. . . */ } Declarative methods No assignment, object creation, or loops Declarative methods allow efficient translation Declarative methods produce compact formulas
Search Algorithm Let S be the states that satisfy rep. Ok() While S is not empty Use a SAT solver Choose a state s in S. Check s. Let P be the set of states similar to s S=S-P Add ¬P to the SAT solver
Approach Program specification Search algorithm State space representation State space reduction Dynamic analysis Static analysis
Dynamic Analysis Discover and prune states that are similar Symbolic execution Generates a path constraint, P P holds for states that traverse the same code path P is the set of similar states to be pruned insert(3, x) 5 2 1 6 4
Dynamic Analysis Discover and prune states that are similar Symbolic execution Generates a path constraint, P P holds for states that traverse the same code path P is the set of similar states to be pruned op(key, value) Operation is insert n 1 n 2 n 4 n 3 n 5 op = insert && root = n 1 && key < n 1. key && n 1. left = n 2 && key > n 2. key && n 2. right = n 5 && key < n 5. key && n 5. left = null Node exists, is greater/less than key Final node does not exist (yet)
Static Analysis Dynamic analysis finds P, the similar states Pruning these states is not always correct! class Why. Static. Analysis { boolean a, b; a = true a = false void operation() { if (a) a = false; else a = true; } b = true @Declarative boolean rep. Ok() { return a || b; } a = true a = false b = false P : = a = true } We use a static analysis to ensure correctness All pruned transitions checked together Any error within finite bounds is caught
Static Analysis class Why. Static. Analysis { boolean a, b; a = false b = true Prestate Poststate of a pruned transition a = true a = false b b void operation() { if (a) a = false; else a = true; } @Declarative boolean rep. Ok() { return a || b; } } Invariant Prestate Invariant Poststate Invariant Correct Transition a = true rep. Ok P : = a = true = (a || b) rep. Ok. Pre = (a || b)a=true = true rep. Ok. Post = (a || b)a=false = b rep. Ok. Pre ® rep. Ok. Post =b Not valid when b = false!
Static Analysis For every valid prestate in P, the following hold The invariant is maintained in the poststate Equality of abstractions rep. Okpre ® rep. Okpost && abs_post. equal. To(abs_post') Operation pre post Use a SAT solver to check If it holds then pruning is sound If not, we have a counterexample Abstraction boolean formula abs post' abs pre Operation abs post Equal
Outline Motivation Example Approach Experimental Results Related Work Conclusions
Checking Modules vs Abstractions Tree. Map Hash. Map Implemented with a red-black tree Implemented with a hash table Abstract. Map Implemented with a linked list of (key, value) pairs
Checking Modules vs Abstractions Tree. Set Hash. Set Implemented with a Tree. Map Implemented with a Hash. Map Abstract. Set Implemented with a linked list of set items
Maps vs Abstract. Map Benchmark Max Number of Nodes 1. 218 5. 556 memory out 0. 608 0. 613 0. 676 0. 732 … 2202. 31 timeout aborted Tree. Map vs Abstract. Map 1 2 3 4 … 8 9 … 15 31 63 0. 188 0. 244 0. 392 0. 485 … 1. 124 1. 491 … 4. 571 40. 405 787. 411 0. 674 6. 514 memory out 0. 465 0. 497 0. 539 0. 810 … 150. 932 1203. 986 timeout aborted Hash. Map vs Abstract. Map 1 2 3 4 … 10 11 12 … 16 32 64 0. 176 0. 227 0. 256 0. 305 … 0. 780 0. 953 1. 162 … 2. 879 75. 139 2004. 723 JPF Korat Blast Glass Box Checker We check over 235 trees in under 15 minutes
Sets vs Abstract. Set Benchmark Max Number of Nodes 0. 537 0. 950 26. 816 memory out 0. 638 0. 648 0. 693 1. 020 … 615. 524 1706. 41 timeout aborted Tree. Set vs Abstract. Set 1 2 3 4 … 13 14 15 … 31 63 0. 195 0. 246 0. 393 0. 489 … 3. 065 3. 399 4. 481 … 39. 789 796. 955 0. 728 0. 781 6. 574 memory out 0. 520 0. 511 0. 716 0. 570 … 238. 420 593. 045 952. 317 timeout aborted Hash. Set vs Abstract. Set 1 2 3 4 … 13 14 15 16 … 32 64 0. 171 0. 221 0. 248 0. 299 … 1. 344 1. 608 2. 029 2. 816 … 68. 011 2543. 034 JPF Korat Blast Glass Box Checker
Checking Clients Benchmark Max Size Original Program Maps Replaced with Abstract. Map Integer. Counter 1 2 3 … 7 15 31 63 127 255 0. 198 0. 279 0. 469 … 1. 182 5. 591 41. 5 632. 488 timeout 0. 113 0. 154 0. 164 … 0. 267 0. 539 1. 867 10. 794 93. 276 946. 091 Dual. Cache 1 2 3 … 7 15 31 63 127 255 0. 203 0. 283 0. 503 … 1. 207 5. 765 53. 434 723. 267 timeout 0. 222 0. 323 0. 327 … 0. 529 1. 015 4. 057 26. 192 215. 521 2180. 506
Checking Clients Benchmark Max Number of Nodes Original Program Maps Replaced with Abstract. Map Tree. Set 1 2 3 … 7 15 31 63 127 255 511 0. 195 0. 246 0. 393. . . 0. 961 4. 481 39. 789 796. 955 timeout 0. 122 0. 153 0. 167 … 0. 251 0. 429 0. 991 3. 388 16. 690 184. 827 425. 328 Hash. Set 1 2 3 4 5 … 16 32 64 128 256 512 0. 171 0. 221 0. 248 0. 299 0. 363 … 2. 816 68. 011 2543. 034 timeout 0. 106 0. 141 0. 153 0. 169 0. 193 … 0. 451 0. 989 3. 464 17. 071 91. 629 754. 426
Related Work State space reduction techniques Abstraction & refinement [SLAM; Blast; Magic] Partial order reduction [Godefroid 97; Flanagan 05] Heap canonicalization [Musuvathi 05; Iosif 02] Symmetry reduction [Ip 93]
Related Work Software model checkers Verisoft [Godefroid 97] Java Pathfinder [Visser 00] CMC [Musuvathi 02] Bandera [Corbett 00] Bogor [Dwyer 05] SLAM [Ball 01] Blast [Henzinger 02] Magic [Chaki 03] Jalloy [Vaziri 03] Miniatur [Dolby 07]
Conclusions Modular Glass Box Model Checking Offers: Significant improvement over traditional model checkers for checking complex data dependent properties A promising approach to checking much larger programs and broader classes of program properties than is currently possible
- Slides: 54