Program Analysis and Verification 0368 4479 Noam Rinetzky

Program Analysis and Verification 0368 -4479 Noam Rinetzky Lecture 11: Interprocedural Shape Analysis Slides credit: Roman Manevich, Mooly Sagiv, Eran Yahav

Canonical Abstraction Top u 1 n u 2 n u 3 [Sagiv, Reps, Wilhelm, TOPLAS 02] 2

Canonical Abstraction Top u 1 n u 2 n u 3 Top 3

Canonical Abstraction (� ) • Merge all nodes with the same unary predicate values into a single summary node • Join predicate values �’(u’ 1 , . . . , u’k) = {� (u 1 , . . . , uk) | f(u 1)=u’ 1 , . . . , f(uk)=u’k } • Converts a state of arbitrary size into a 3 -valued abstract state of bounded size • � (C) = �{ � (c) | c �C } 4

Information Loss Top n n Canonical abstraction Top Top n n n Top 5

Instrumentation Predicates • Record additional derived information via predicates rx(v) = v 1: x(v 1) n*(v 1, v) c(v) = � v 1: n(v 1, v) n*(v, v 1) Top n Canonical abstraction n Top n c Top c 6

Embedding Theorem: Conservatively Observing Properties Top r. Top No Cycles �� v 1, v 2: n(v 1, v 2) n*(v 2, v 1)1/2 r. Top No cycles (derived) � v: � c(v) 1 7

Operational Semantics void push (int v) { Node x = malloc(sizeof(Node)); x->d = v; x->n = Top; Top = x; } int pop() { Top n u 1 u 2 n u 3 if (Top == NULL) return EMPTY; Node *s = Top->n; � s = Top->n� s’(v) = v 1: Top(v 1) n(v 1, v) int r = Top->d; Top = s; return r; } Top n u 1 s u 2 n u 3 8

Abstract Semantics s = Top n Top r. Top ? s Top � s = Top->n� s’(v) = v 1: Top(v 1) n(v 1, v) r. Top 9

Best Transformer (s = Top n) s Top r. Top Concrete Semantics r. Top . . . r. Top s’(v) = v 1: Top(v 1) n(v 1, v) r. Top . . . r. Top Canonical Abstraction ? Abstract Semantics Top r. Top s � r. Top s Top r. Top s r. Top 10

Semantic Reduction • Improve the precision of the analysis by recovering properties of the program semantics • A Galois connection (C, , , A) • An operation op: A A is a semantic reduction when – l L 2 op(l) l and – (op(l)) = (l) l op C A

The Focus Operation • Focus: Formula (� (3 -Struct) � (3 -Struct)) • Generalizes materialization • For every formula – Focus( )(X) yields structure in which evaluates to a definite values in all assignments – Only maximal in terms of embedding – Focus( ) is a semantic reduction – But Focus( )(X) may be undefined for some X

Partial Concretization Based on Transformer (s=Top n) s Top r. Top Abstract Semantics r. Top s Top r. Top Partial Concretization r. Top s’(v) = v 1: Top(v 1) n(v 1, v) r. Top Canonical Abstraction Focus (Top� n) u: top(u) n(u, v) s Abstract Semantics Top Top r. Top s Top r. Top 13

Partial Concretization • Locally refine the abstract domain per statement • Soundness is immediate • Employed in other shape analysis algorithms [Distefano et. al. , TACAS’ 06, Evan et. al. , SAS’ 07, POPL’ 08] 14

The Coercion Principle • Another Semantic Reduction • Can be applied after Focus or after Update or both • Increase precision by exploiting some structural properties possessed by all stores (Global invariants) • Structural properties captured by constraints • Apply a constraint solver

Apply Constraint Solver Top r. Top x r. Top n x rx n y rx, ry r. Top n n n rx, ry n x rx n y rx, ry n rx, ry

Sources of Constraints • Properties of the operational semantics • Domain specific knowledge – Instrumentation predicates • User supplied

Example Constraints x(v 1) x(v 2) eq(v 1, v 2) n(v, v 1) n(v, v 2) eq(v 1, v 2) n(v 1, v) n(v 2, v) eq(v 1, v 2) is(v) n*(v 1, v 2) t[n](v 1, v 2)

Abstract Transformers: Summary • Kleene evaluation yields sound solution • Focus is a statement-specific partial concretization • Coerce applies global constraints

DEMO

What about procedures? 21

A Semantics for Procedure Local Heaps and its Abstractions Noam Rinetzky Tel Aviv University Jörg Bauer Universität des Saarlandes Thomas Reps University of Wisconsin Mooly Sagiv Tel Aviv University Reinhard Wilhelm Universität des Saarlandes

Motivation • Interprocedural shape analysis – Conservative static pointer analysis – Heap intensive programs • Imperative programs with procedures • Recursive data structures • Challenge – Destructive update – Localized effect of procedures

Main idea • Local heaps x x y call p(x); y g g t t

Main idea • Local heaps • Cutpoints x x y call p(x); y g g t t

Main Results • Concrete operational semantics – Large step • Functional analysis – Storeless • Shape abstractions – Local heap – Observationally equivalent to “standard” semantics • Java and “clean” C • Abstractions – Shape analysis [Sagiv, Reps, Wilhelm, TOPLAS ‘ 02] – May-alias [Deutsch, PLDI ‘ 94] –…

Outline • Motivating example – Local heaps – Cutpoints • Why semantics • Local heap storeless semantics • Shape abstraction

Example static List reverse(List t) { static void main() { … t p qq p n n List x = reverse(p); n n n x t List y = reverse(q); List z = reverse(x); } n n n return r; } r

Example static List reverse(List t) { static void main() { List x = reverse(p); p t q n n n t n n x List y = reverse(q); q n n n y t List z = reverse(x); } n n n return r; } r

Example n q List y = reverse(q); n n n t n static List reverse(List t) { static void main() { List x = reverse(p); p n t n n t x y List z = reverse(x); p x n n n z t n n n return r; } } r

Cutpoints • Separating objects – Not pointed-to by a parameter

Cutpoints • Separating objects – Not pointed-to by a parameter proc(x) p n n n x Stack sharing

Cutpoints • Separating objects – Not pointed-to by a parameter proc(x) p n n n proc(x) n x Stack sharing n n x y Heap sharing

Cutpoints • Separating objects – Not pointed-to by a parameter • Capture external sharing patterns proc(x) p n n n proc(x) n x Stack sharing n n x y Heap sharing

Example static List reverse(List t) { static void main() { List x = reverse(p); List y = reverse(q); n n n p n q n n t x y List z = reverse(x); p z n n n x r n n n return r; } } t

Outline ü Motivating example • Why semantics • Local heap storeless semantics • Shape abstraction

Abstract Interpretation [Cousot and Cousot, POPL ’ 77] Operational semantics Abstract transformer

Introducing local heap semantics Operational semantics ~ Local heap Operational semantics ’ ’ Abstract transformer

Outline ü Motivating example ü Why semantics • Local heap storeless semantics • Shape abstraction

Programming model • Single threaded • Procedures üValue parameters üRecursion • Heap üRecursive data structures üDestructive update No explicit addressing (&) No pointer arithmetic

Simplifying assumptions • No primitive values (only references) • No globals • Formals not modified

Storeless semantics • No addresses • Memory state: – Object: – Heap: 2 Object x x n x. n. n y=x 2 Access paths x y n • Alias analysis x. n n y. n x. n. n y. n. n x=null y y n y. n. n

Example static void main() { List x = reverse(p); static List reverse(List t) { List y = reverse(q); p x. n. n. n t. n. n. n p q y. n. n q n n t. n. n x. n. n y. n n t. n x. n n y n xt t. n. n. n t x n t. n. n n t t y List z = reverse(x); z z n z. n. n n x z. n. n. n x r } p? } r return r; n r. n. n n tt r. n. n. n t

Example static void main() { List x = reverse(p); static List reverse(List t) { L List y = reverse(q); L t. n. n. n p x. n. n. n p. L n y. n. n q t. n. n x. n. n y. n n n t. n x. n y n xt t. n. n. n n t. n. n L t x n t. n n t t y List z = reverse(x); p z pp z n p. n z. n n p. n. n z. n. n nx p. n. n. n z. n. n. n x L r } } L r return r; n L. n r. n n L. n. n r. n. n n t L. n. n. n r. n. n. n

Cutpoint labels • Relate pre-state with post-state • Additional roots • Mark cutpoints at and throughout an invocation

Cutpoint labels • Cutpoint label: the set of access paths that point to a cutpoint – when the invoked procedure starts L t. n. n. n L t. n. n t. n L {t. n. n. n} t t

Sharing patterns • Cutpoint labels encode sharing patterns L t. n. n. n L n t. n. n n t t L n t. n. n. n L n p Stack sharing t. n. n w. n n n t. n w Heap sharing L {t. n. n. n} t t

Observational equivalence • L L (Local-heap Storeless Semantics) • G G (Global-heap Store-based Semantics) L and G observationally equivalent when for every access paths AP 1, AP 2 �AP 1 = AP 2 � ( L) �AP 1 = AP 2 � ( G )

Main theorem: semantic equivalence • L L (Local-heap Storeless Semantics) • G G (Global-heap Store-based Semantics) • L and G observationally equivalent st, L � ’L � st, G � ’G ’L and ’G are observationally equivalent LSL GSB

Corollaries • Preservation of invariants – Assertions: AP 1 = AP 2 • Detection of memory leaks

Applications • Develop new static analyses – Shape analysis • Justify soundness of existing analyses

Related work • Storeless semantics – Jonkers, Algorithmic Languages ‘ 81 – Deutsch, ICCL ‘ 92

Shape abstraction • Shape descriptors represent unbounded memory states – Conservatively – In a bounded way • Two dimensions – Local heap (objects) – Sharing pattern (cutpoint labels)

A Shape abstraction L={t. n. n. n} r L n r. n L. n n r. n. n L. n. n n t, r. n. n. n L. n. n. n t

A Shape abstraction L=* r L n r. n L. n n r. n. n L. n. n n t, r. n. n. n L. n. n. n t

A Shape abstraction L=* r L n r r LL n n r. n L. n nn r. n L. n t, t, r. n L. n t

A Shape abstraction L=* r L n r. n L. n n t, r. n L. n t

A Shape abstraction L={t. n. n. n} r L n r. n L. n n r. n. n L=* r L n t, r. n. n. n L. n. n. n t t, r. n L. n t n r L n r. n L. n n

A Shape abstraction L 1={t. n. n. n} d d n L 2 r r n L 1 d. n L 2. n r. n L 1. n L 2={g. n. n. n} n n d. n. n L 2. n. n n r. n. n L 1. n. n L=* d g t, r. n. n. n L 1. n. n. n t t, d. n L. n t t, r. n L. n t n d L L r n g, d. n. n. n L 2. n. n. n r L n n d. n L. n r. n L. n n

Cutpoint-Freedom 63

How to tabulate procedures? • Procedure input/output relation – Not reachable Not effected – proc: local ( reachable) heap local heap main() { x n n t append(y, z); p q p y z append(List p, List q) { … } q p n n n y } z q

How to handle sharing? • External sharing may break the functional view main()p{ y n n t y x append(y, z); n n x } q z n z p n n q append(List p, List q) { … } q p n n n

What’s the difference? 1 st Example x t n 2 nd Example n y y append(y, z); z t n n x append(y, z); z

Cutpoints • An object is a cutpoint for an invocation – Reachable from actual parameters – Not pointed to by an actual parameter – Reachable without going through a parameter append(y, z) y t n append(y, z) n y x z t n n z

Cutpoint freedom • Cutpoint-free – Invocation: has no cutpoints – Execution: every invocation is cutpoint-free – Program: every execution is cutpoint-free append(y, z) n y x append(y, z) n t x z t n n y z

Interprocedural shape analysis for cutpoint-free programs using 3 -Valued Shape Analysis

Memory states: 2 -Valued Logical Structure • A memory state encodes a local heap – Local variables of the current procedure invocation – Relevant part of the heap • Relevant Reachable main append q p x t n n y z

Memory states • Represented by first-order logical structures Predicate x(v) n(v 1, v 2) Meaning Variable x points to v Field n of object v 1 points to v 2

Memory states • Represented by first-order logical structures Predicate x(v) n(v 1, v 2) p nu 1 q u 2 Meaning Variable x points to v Field n of object v 1 points to v 2 p u 1 1 u 2 0 q u 1 0 u 2 1 n u 1 u 2 u 1 0 1 u 2 0 0

Operational semantics • Statements modify values of predicates • Specified by predicate-update formulae – Formulae in FO-TC

Procedure calls append(p, q) x z y n 1. Verify cutpoint freedom 2 Compute input append(y, z) … Execute callee … 3 Combine output x z y n n q p append body q p n

Procedure call: 1. Verifying cutpoint-freedom • An object is a cutpoint for an invocation – Reachable from actual parameters – Not pointed to by an actual parameter – Reachable without going through a parameter append(y, z) n y t x n n Cutpoint free y z t n n n x Not Cutpoint free z

Procedure call: 1. Verifying cutpoint-freedom • Invoking append(y, z) in main – R{y, z}(v)= v 1: y(v 1) n*(v 1, v) v 1: z(v 1) n*(v 1, v) – is. CPmain, {y, z}(v)= R{y, z}(v) ( y(v) z(v 1)) ( x(v) t(v) v 1: R{y, z}(v 1) n(v 1, v)) (main’s locals: x, y, z, t) n y t x n n Cutpoint free y z t n n n x Not Cutpoint free z

Procedure call: 2. Computing the input local heap • Retain only reachable objects • Bind formal parameters Call state n y t x n Input state n p z n n q

Procedure body: append(p, q) Input state p n Output state n p q n n n q

Procedure call: 3. Combine output Output state Call state n y x t n n p z n n n q

Procedure call: 3. Combine output Output state Call state n y x t n p z n Auxiliary predicates in. Uc(v) in. Ux(v) n n n y x t n n q n n z

Observational equivalence • CPF (Cutpoint free semantics) • GSB (Standard semantics) CPF and GSB observationally equivalent when for every access paths AP 1, AP 2 �AP 1 = AP 2 � ( CPF) �AP 1 = AP 2 � ( GSB)

Observational equivalence • For cutpoint free programs: – CPF (Cutpoint free semantics) – GSB (Standard semantics) – CPF and GSB observationally equivalent • It holds that – st, CPF � ’CPF � st, GSB � ’GSB – ’CPF and ’GSB are observationally equivalent

Introducing local heap semantics Operational semantics ~ Local heap Operational semantics ’ ’ Abstract transformer

Shape abstraction • Abstract memory states represent unbounded concrete memory states – Conservatively – In a bounded way – Using 3 -valued logical structures

3 -Valued logic • • 1 = true 0 = false 1/2 = unknown A join semi-lattice, 0 � 1 = 1/2

Canonical abstraction y x t n n z y x n n n t n n n

Instrumentation predicates • Record derived properties • Refine the abstraction – Instrumentation principle [SRW, TOPLAS’ 02] • Reachability is central! Predicate Meaning rx(v) v is reachable from variable x robj(v 1, v 2) v 2 is reachable from v 1 ils(v) v is heap-shared c(v) v resides on a cycle

Abstract memory states (with reachability) y rx x t n n rt rx rt rt n n rx rx, ry rz rz n n rz rz rt n rx n rt n z y n rx, ry z rz n n rz rz rz

The importance of reachability: Call append(y, z) z y rx x t rx rt rx n rt t x n n n rt rx n n n rx rx, ry rz n rz rz rt y z rx, ry rz n n rz x n rt n n n y z n n

Abstract semantics • Conservatively apply statements on abstract memory states – Same formulae as in concrete semantics – Soundness guaranteed [SRW, TOPLAS’ 02]

Procedure calls append(p, q) x z y n 1. Verify cutpoint freedom 2 Compute input append(y, z) … Execute callee … 3 Combine output x z y n n q p append body q p n

Conservative verification of cutpointfreedom • Invoking append(y, z) in main – R{y, z}(v)= v 1: y(v 1) n*(v 1, v) v 1: z(v 1) n*(v 1, v) – is. CPmain, {y, z}(v)= R{y, z}(v) ( y(v) z(v 1)) ( x(v) t(v) v 1: R{y, z}(v 1) n(v 1, v)) y t ry n ry rt n n n rz n rt Cutpoint free z y t ry n ry ryn n n rt rt rx rz x z Not Cutpoint free

Interprocedural shape analysis Tabulation exits x p y call f(x) p x y

Interprocedural shape analysis Analyze f p Tabulation exits p x y call f(x) x y

Interprocedural shape analysis • Procedure input/output relation Output Input q q rq rq q p rp n p rp rq n q p rq rp … rp n nr rp q q rp n q nrr p q

Interprocedural shape analysis • Reusable procedure summaries – Heap modularity q p rp y x rx ry n n rx n g rg n y rx ry h rh rq z p rp nr q q rp z append(y, z) rz ik rirk x rx append(h, i) n n n ry rx rz z y x append(y, z) rx n g rg n y rx ry h rgrh ry z nr r r x y z ik n r rg h ri rk

Plan ü Cutpoint freedom ü Non-standard concrete semantics ü Interprocedural shape analysis • Prototype implementation

Prototype implementation • TVLA based analyzer • Soot-based Java front-end • Parametric abstraction Data structure Singly linked list Sorting (of SLL) Unshared binary trees Verified properties Cleanness, acyclicity + Sortedness Cleaness, tree-ness

Iterative vs. Recursive (SLL) 585

Inline vs. Procedural abstraction // Allocates a list of // length 3 List create 3(){ … } main() { List x 1 = create 3(); List x 2 = create 3(); List x 3 = create 3(); List x 4 = create 3(); … }

Call string vs. Relational vs. CPF [Rinetzky and Sagiv, CC’ 01] [Jeannet et al. , SAS’ 04]

Summary • Cutpoint freedom • Non-standard operational semantics • Interprocedural shape analysis – Partial correctness of quicksort • Prototype implementation

Application • Properties proved – Absence of null dereferences – Listness preservation – API conformance • Recursive Iterative • Procedural abstraction

Related Work • Interprocedural shape analysis – – – Rinetzky and Sagiv, CC ’ 01 Chong and Rugina, SAS ’ 03 Jeannet et al. , SAS ’ 04 Hackett and Rugina, POPL ’ 05 Rinetzky et al. , POPL ‘ 05 • Local Reasoning – Ishtiaq and O’Hearn, POPL ‘ 01 – Reynolds, LICS ’ 02 • Encapsulation – Noble et al. IWACO ’ 03 –. . .

Summary • Operational semantics – Storeless – Local heap – Cutpoints – Equivalence theorem • Applications – Shape analysis – May-alias analysis