Shape Analysis with Structural Invariant Checkers BorYuh Evan

Shape Analysis with Structural Invariant Checkers Bor-Yuh Evan Chang Xavier Rival George C. Necula University of California, Berkeley SAS 2007

Example: Typestate with shape analysis Concrete Example Abstraction l l “red list” cur = l; program-specific predicate while (cur != null) { assert(cur is red); heap abstraction flow-sensitive make_purple(cur); make_purple(¢) could be l cur = cur!next; • • lock(¢) “purple “red l free(¢) list segment” list” open(¢) cur … } Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 2

Shape analysis is not yet practical Usability: Usability Choosing the heap abstraction difficult “red list” Space Invader [Distefano et al. ] “red list” - Hard to extend + No additional user effort Parametric in low-level, analyzer-oriented predicates red(n) Æ n 2 reach(l) TVLA [Sagiv et al. ] + Very general and expressive - Hard for non-expert Parametric in high-level, developer-oriented predicates “red list” developer Built-in high-level predicates Our Proposal + Extensible + Easier for developers Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 3

Shape analysis is not yet practical Scalability: Scalability Finding right level of abstraction difficult Over-reliance on disjunction for precision “purple “red list segment” list” l cur developer emp shape analyzer l cur Ç Ç l cur Ç l l, cur Ç Ç Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers Ç l, cur l cur 4

Hypothesis The developer can describe the memory in a compact manner at an abstraction level sufficient for the properties of interest (at least informally). • Good abstraction is program-specific l “purple list segment” “red list” abstraction ideas cur developer ? shape analyzer Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 5

Observation Checking code expresses a shape invariant and an intended usage pattern. bool redlist(List* l) { if (l == null) return true; else return l!color == red && redlist(l!next); l l } Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 6

Proposal An automated shape analysis with a memory abstraction based on invariant checkers bool redlist(List* l) { if (l == null) return true; else return l!color == red && redlist(l!next); } checkers developer shape analyzer • Extensible – Abstraction based on the developer-supplied checkers • Targeted for Usability – Code-like global specification, local invariant inference • Targeted for Scalability – Based on the hypothesis Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 7

Outline • Memory abstraction – Restrictions on checkers – Challenge: Intermediate invariants • Analysis algorithm – Strong updates – Challenge: Ensuring termination • Experimental results Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 8

Abstract memory using checkers “Some number of points-to edges that satisfies checker c” Graphs ® ® ¯ f values (address or null) ® points-to relation (memory cell) ® checker run c c ¯ partial run Example “Disjointly, ®!next = ¯, °!next = ¯, and ¯ is a list. ” ® next ¯ ° list next disjoint memory regions Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 9

Checkers as inductive definitions bool list(List* l) { if (l == null) return true; else return list(l!next); } list(l) Disjointness Checker run can list(…) dereference any object field only once ® : = 9¯. list emp ® = null ® ¯ next emp ® ® next Ç list ® null (® = null) null next null … Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 10

What can a checker do? • In this talk, a checker … – is a pure, recursive function – dereferences any object field only once during a run – only one argument can be dereferenced (traversal arg) Traversal argument bool skip 1(Skip* l) { if (l == null) return true; else { Only fields Skip* s = l!skip; from traversal return skip 0(l!next, s) argument && skip 1(s); } } ® : = 9¯, °. skip 1 emp ® = null Ç skip ® next ¯ skip 0(g) Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers ° skip 1 ® null 11

back to the abstract domain … bool redlist(List* l) { if (l == null) return true; else return l!color == red && redlist(l!next); } checkers shape analyzer

Challenge: Intermediate invariants assert(redlist(l)); cur = l; l redlist while (cur != null) { l purplelist make_purple(cur); Prefix Segment Described cur = cur!next; by ? } assert(purplelist(l)); l cur redlist Suffix Described by checkers purplelist Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 13

Prefix segments as partial checker runs Abstraction Checker Run l purplelist a cur c purplelist(l) c(a) purplelist(…) c(…) purplelist(cur) c(…) c(…) Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 14

Outline • Memory abstraction – Restrictions on checkers – Challenge: Intermediate invariants • Analysis algorithm – Strong updates – Challenge: Ensuring termination • Experimental results Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 15

Flow function: Unfold and update edges x!next = x!next; x Unfold inductive definition x next list materialize: x!next, x!next Ç x Strong updates using disjointness of regions next list update: x!next = x!next x next list Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 16

Challenge: Termination and precision last = l; cur = l!next; Observation while (curiterates != null) { Previous // “less … cur, unfolded” last … are if (…) last = cur; cur = cur! next; } Fold into checker edges But where and how much? next l, last l l next cur last list next cur list widen (canonicalize, blur) l list last next list Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers cur list 17

last = l; cur = l!next; while (cur != null) { if (…) last = cur; cur = cur! next; } History-guided folding • Match edges to identify where to fold • Apply local folding rules l next l, last l next cur last list next cur list l, last v l list ? last ? Yes l list last next cur list Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 18

Summary: Enabling checker-based shape analysis • Built-in disjointness of memory regions – As in separation logic – Checkers read any object field only once in a run • Generalized segment abstraction – Based on partial checker runs a c • Generalized folding into inductive predicates – Based on iteration history (i. e. , a widening operator) l, cur list l next cur list l list Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers cur list 19

Outline • Memory abstraction – Restrictions on checkers – Challenge: Intermediate invariants • Analysis algorithm – Strong updates – Challenge: Ensuring termination • Experimental results Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 20

Experimental results Benchmark Lines of Code Analysis Time Max. Num. Graphs at a Program Point Max. Num Iterations at a Program Point list reverse 019 0. 007 s 1 03 list remove element 027 0. 016 s 4 06 list insertion sort 056 0. 021 s 4 07 search tree find 023 0. 010 s 2 04 skip list rebalance 033 0. 087 s 6 07 scull driver 894 9. 710 s 4 16 • Verified structural invariants as given by checkers are preserved across data structure manipulation • Limitations (in scull driver) – Arrays not handled (rewrote as linked list), char arrays ignored • Promising as far as number of disjuncts Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 21

Conclusion • Invariant checkers can form the basis of a memory abstraction that – Is easily extensible on a per-program basis – Expresses developer intent • Critical for usability • Prerequisite for scalability • Start with usability • Work towards expressivity Chang, Rival, Necula - Shape Analysis with Structural Invariant Checkers 22

What can checker-based shape analysis do for you?