Static Program Analysis via ThreeValued Logic Thomas Reps

Static Program Analysis via Three-Valued Logic Thomas Reps University of Wisconsin Joint work with M. Sagiv (Tel Aviv) and R. Wilhelm (U. Saarlandes)

x = 3; y = 1/(x-3); need to track values other than 0 x = 3; px = &x; y = 1/(*px-3); need to track heap-allocated storage need to track pointers x = 3; p = (int*)malloc(sizeof int); *p = x; q = p; y = 1/(*q-3);

a = &e 1 b = a 2 c = &f 3 *b = c 4 d = *a 5 a b c d a b c d e Flow-Sensitive Points-To Analysis f e f p q p = &q; p q e p f e p r 1 r 2 s 1 s 2 s 3 r 2 r 1 s 1 r 1 q p = q; p q p = *q; p r 1 r 2 q s 1 s 2 s 3 r 2 r 1 s 1 r 1 q f e f p r 2 s 2 q *p = q; p r 2 s 2 q

What About Malloc? • Each malloc site malloc-site variable int *p, *q, *r; p a: p = (int*)malloc(sizeof(int)); b: q = (int*)malloc(sizeof(int)); r = p; r q malloc$a malloc$b

What About Malloc? • Each malloc site malloc-site variable typedef struct list_cell { int val; struct list_cell *next; } *List; head p /* Create a List of length n */ List head; List *p = &head; for (int i = 0; i < n; i++) { *p = (List)malloc(sizeof(List*)); p = &((*p) next); }

Shape Analysis [Jones and Muchnick 1981] • Characterize dynamically allocated data – Identify may-alias relationships – x points to an acyclic list, tree, dag, … – “disjointedness” properties • x and y point to structures that do not share cells – show that data-structure invariants hold • Account for destructive updates through pointers

Applications: Software Tools • Static detection of memory errors – dereferencing NULL pointers – dereferencing dangling pointers – memory leaks • Static detection of logical errors – Is a data-structure invariant restored?

Why is Shape Analysis Difficult? • Destructive updating through pointers – p next = q – Produces complicated aliasing relationships • Dynamic storage allocation – No bound on the size of run-time data structures • Data-structure invariants typically only hold at the beginning and end of operations – Want to verify that data-structure invariants are re-established

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t NULL y 1 2 x 3 NULL

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x Materialization

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Example: In-Situ List Reversal typedef struct list_cell { int val; struct list_cell *next; } *List; List reverse (List x) { List y, t; y = NULL; while (x != NULL) { t = y; y = x; x = x next; y next = t; } return y; } t y NULL x

Idea for a List Abstraction t y x t NULL y x represents t y x NULL

t x != NULL y t y NULL x t t=y x y t y NULL t x = x next t y x y t y x x t x y NULL t y x t x y t y x NULL return y y NULL y next = t x t NULL x y=x y t NULL x x x

Properties of reverse(x) • On entry, x points to an acyclic list • On each iteration, x & y point to disjoint acyclic lists • All the pointer dereferences are safe • No memory leaks • On exit, y points to an acyclic list • On exit, x = = NULL • All cells reachable from y on exit were reachable from x on entry, and vice versa • On exit, the order between neighbors in the y-list is opposite to their order in the x-list on entry

A ‘Yacc’ for Shape Analysis: TVLA • Parametric framework – Some instantiations known analyses – Other instantiations new analyses • Applications beyond shape analysis – – – Partial correctness of sorting algorithms Safety of mobile code Deadlock detection in multi-threaded programs Partial correctness of mark-and-sweep gc alg. Correct usage of Java iterators

A ‘Yacc’ for Static Analysis: TVLA • Parametric framework – Some instantiations known analyses – Other instantiations new analyses • Applications beyond shape analysis – – – Partial correctness of sorting algorithms Safety of mobile code Deadlock detection in multi-threaded programs Partial correctness of mark-and-sweep gc alg. Correct usage of Java iterators

Formalizing “. . . ” Informal: x Formal: x Summary node

The French Recipe for Program Verification [Cousot & Cousot] • Concrete operational semantics – st : • Collecting semantics – st c: 2 2 • Abstract semantics – : 2 A, : A 2 , ( (a)) = a, ( (C)) C – st #: A A – Upper approximation • Sound results • But may produce false alarms

Using Relations to Represent Linked Lists

Using Relations to Represent Linked Lists x y u 1 u 2 u 3 u 4

Formulas: Queries for Observing Properties Are x and y pointer aliases? v: x(v) y(v)

Are x and y Pointer Aliases? Yes v: x(v) y(v) 1 = x y u 1 u 2 u 3 u 4

Predicate-Update Formulas for “y = x” • x’(v) = x(v) • y’(v) = x(v) • t’(v) = t(v) • n’(v 1, v 2) = n(v 1, v 2)

Predicate-Update Formulas for “y = x” y’(v) = x(v) x y 1 0 0 0 u 1 u 2 u 3 u 4

Predicate-Update Formulas for “x = x n” • x’(v) = v 1: x(v 1) n(v 1, v) • y’(v) = y(v) • t’(v) = t(v) • n’(v 1, v 2) = n(v 1, v 2)

Predicate-Update Formulas for “x = x n” x’(v) = v 1: x(v 1) n(v 1, v) x y 0 1 u 2 u 3 u 4

Why is Shape Analysis Difficult? • Destructive updating through pointers – p next = q – Produces complicated aliasing relationships • Dynamic storage allocation – No bound on the size of run-time data structures • Data-structure invariants typically only hold at the beginning and end of operations – Need to verify that data-structure invariants are re-established

Two- vs. Three-Valued Logic Two-valued logic 0 Three-valued logic 1 {0, 1} {0} {1} {0} {0, 1} {1} {0, 1}

Two- vs. Three-Valued Logic Two-valued logic Three-valued logic

Two- vs. Three-Valued Logic Two-valued logic 0 Three-valued logic 1 {0, 1} {0} {1}

Two- vs. Three-Valued Logic Two-valued logic 0 Three-valued logic 1 ½ 0 1 0 ½ 1 ½

Boolean Connectives [Kleene]

Canonical Abstraction x u 1 u 2 u 3 u 4 0 0 1 0 0 0 x u 1 u 234

Canonical Abstraction x u 1 u 2 u 3 u 4 x u 1 u 234

Canonical Abstraction x u 1 u 2 u 3 u 4 0 1 x u 1 u 234

Property-Extraction Principle • Questions about a family of two-valued stores can be answered conservatively by evaluating a formula in a three-valued store • Formula evaluates to 1 formula holds in every store in the family • Formula evaluates to 0 formula does not hold in any store in the family • Formula evaluates to 1/2 formula may hold in some; not hold in others

Are x and y Pointer Aliases? x y u 1 u v: x(v) y(v) 1 1 1 Yes

Is Cell u Heap-Shared? Yes u v 1, v 2: n(v 1, u) n(v 2, u) v 1 v 2 1 1

Is Cell u Heap-Shared? x y Maybe u u 1 v 1, v 2: n(v 1, u) n(v 2, u) v 1 v 2 1/2 1/2 1

The Embedding Theorem x No y v: x(v) y(v) No No Maybe u 1 y u 34 u 2 x u 1 x u 4 u 3 u 2 x u 234 y u 1234

The Embedding Theorem • If a structure B can be embedded in a structure S by an onto function f, such that basic predicates are preserved, i. e. , p. B(u 1, . . , uk) p. S (f(u 1), . . . , f(uk)) • Then every formula is preserved: – If = 1 in S, then = 1 in B – If = 0 in S, then = 0 in B – If = 1/2 in S, then could be 0 or 1 in B

Embedding x x u 1 u 2 x u 456 u 123 u 12 u 4 u 34 u 56 u 6

Canonical Abstraction: An Embedding Whose Result is of Bounded Size x u 1 u 2 u 3 u 4 x u 1 u 234

Predicate-Update Formulas for “y = x” y’(v) = x(v) Old: New: x x y u 1 u 1 u

Predicate-Update Formulas for “x = x n” x’(v) = v 1: x(v 1) n(v 1, v) New: Old: x y u u 1 0 1/2 y u 1 x u

t x != NULL 1 0 y NULL x t t=y y x u 1 NULL u x t y=x y y’(v) = x(v) NULL x x = x next y next = t return y x y u 1 u

t x != NULL y t y NULL x t t=y x y t y NULL t x = x next t y x y t y x x t x y NULL t y x t x y t y x NULL return y y NULL y next = t x t NULL x y=x y t NULL x x x

Naïve Transformer (x = x n) x y 0 1/2 Evaluate update formulas x y x’(v) = v 1: x(v 1) n(v 1, v)

Cyclic versus Acyclic Lists x 31 x u 1 71 u 91

How Are We Doing? • Conservative • Convenient • But not very precise – Advancing a pointer down a list loses precision – Cannot distinguish an acyclic list from a cyclic list

The Instrumentation Principle • Increase precision by storing the truth-value of some chosen formulas

Is Cell u Heap-Shared? u v 1, v 2: n(v 1, u) n(v 2, u) v 1 v 2

Example: Heap Sharing is(v) = v 1, v 2: n(v 1, v) n(v 2, v) v 1 v 2 x 31 71 91 is = 0 x u 1 is = 0 u is = 0

Example: Heap Sharing is(v) = v 1, v 2: n(v 1, v) n(v 2, v) v 1 v 2 x 31 71 91 is = 0 is = 10 is = 0 x u 1 u is = 0 is = 1 is = 0

Example: Cyclicity c(v) = v 1: n(v, v 1) n*(v 1, v) x 31 71 91 c=0 c == 10 is c=1 x u 1 c=0 u c=1

Is Cell u Heap-Shared? is = 0 x y No! is = 0 u u 1 v 1, v 2: n(v 1, u) n(v 2, u) v 1 v 2 1/2 1/2 1 Maybe

Formalizing “. Informal: x y Formal: x y . . ”

Formalizing “. Informal: t 1 x y Formal: t 2 x t 1 y t 2 . . ”

Formalizing “. . . ” Informal: x y Formal: reachable from variable x reachable from variable y x r[x] r[y] y

Formalizing “. Informal: . . ” t 1 x y t 2 Formal: x r[x] r[y] t 1 r[x], r[t 1] r[y], r[t 2] y t 2

Updating Auxiliary Information x y t 1 }p[x], p[y]} }r[x], r[y]} }p[t 1], r[x], r[y]} }r[t 1], r[x], r[y]} y = NULL t 1 x }p[x]} }r[x]} }p[t 1], r[x]} }r[t 1], r[x]}

Automatic Generation of Update Formulas for Instrumentation Predicates Where do we get the predicate-update formulas to update the extra predicates?

Automatic Generation of Update Formulas for Instrumentation Predicates • Originally, user provided c, st(v) – Definitions of instrumentation predicates: p(v) – Update formulas for instrumentation predicates: p, st(v) – Update formulas for core predicates: Consistently defined? Now: p, st created from p and the c, st d d

Useful Instrumentation Predicates • • • doubly-linked(v) reachable-from-variable-x(v) acyclic-along-dimension-d(v) tree(v) dag(v) AVL trees: Need FO + TC – balanced(v), left-heavy(v), right-heavy(v) –. . . but not via height arithmetic

Materialization Formal: x y x x = x n y Informal: x y x x = x n y Formal: x y x x = x n y

Naïve Transformer (x = x n) x y Evaluate update formulas x y

Best Transformer (x = x n) x y x y . . . Evaluate update formulas x y y x . . . x y y x

“Focus”-Based Transformer (x = x n) x y Focus(x n) x y “Partial ” Evaluate update formulas x y y x