Comparing Semantic and Syntactic Methods in Mechanized Proof

Comparing Semantic and Syntactic Methods in Mechanized Proof Frameworks C. J. Bell, Robert Dockins, Aquinas Hobor, Andrew W. Appel, David Walker 1

• In the last decade, dozens of researchers have been investigating proof-carrying code (PCC) • These researchers have split into two camps: – those using syntactic proof methods – those using semantic proof methods 2

List-Machine Benchmark • We want to be able to investigate different proof methodologies, such as syntactic and semantic type systems • The list-machine benchmark is – – assembly language operational semantics type system specification two implementaions of a type system • This benchmark is – simple, so that it is easy to understand – modular, so that it is flexible – publically available at • http: //www. cs. princeton. edu/~appel/listmachine/2. 0 3

Changes to the List-Machine Benchmark for 2. 0 • Implemented only in Coq • Added a semantic type system • Reorganized the framework 4

Outline ü Introduction • Organization of the List-Machine framework • Extend the List Machine with fault tolerance • Semantic and syntactic methods in large systems 5

Machine Specification 6

Modules 7

Modules Type System Proves: Π⊢blocks Ψ → safe Ψ Typechecking Algorithm check(Π, Ψ) = true Typechecking Algorithm Type System Specification • type operators • definitions of typing rules • statement of safety • Π⊢blocks Ψ → safe Ψ Typechecker Soundness Proof check(Π, Ψ) = true → Π ⊢blocksΨ Typechecker Type System Soundness Proof Specification 8

Type System Proves: Π⊢blocks Ψ → safe Ψ Type System Specification • type operators • definitions of typing rules • statement of safety • Π⊢blocks Ψ → safe Ψ Typechecking Algorithm check(Π, Ψ) = true Typechecker Soundness Proof check(Π, Ψ) = true → Π ⊢blocksΨ 9

Syntactic Type System Specification Syntactic Soundness Proof Π⊢blocks Ψ → safe Ψ • Type operators defined inductively • Typing rules defined inductively • The type system is proven sound using metatheorems (progress & preservation) using induction over definitions. 10

Semantic Type System Specification Semantic Soundness Proof Π⊢blocks Ψ → safe Ψ List Machine Hoare Logic Π⊢blocks Ψ Π; Ψ⊢block ι: P Π; Ψ⊢instr P{ι}Q Modal Specification Logic Modal Model Library reusable 11

Outline ü Introduction ü Organization of the List-Machine framework • Extend the List Machine with fault tolerance • Semantic and syntactic methods in large systems 12

Fault Tolerance • Extend the List-Machine framework to provide fault tolerance – Requires non-trivial modifications to the framework – Demonstrates the flexibility of the framework 13

Simple List-Machine Example (without faults) 14

Fault Model • Single Event Upset – assume a fault will occur at most once • A fault may change just one register’s value to any other value.

Simple List-Machine Example (with faults) 16

Fault-Tolerant Modified Machine Specification 17

Fault-Tolerant Example 18

Incorrect Fault-Tolerant Example 19

Is the modified code fault-tolerant? • Fault tolerance becomes part of the safety property • Type system ensures proper use of colors • Model possible occurrences of faults 20

Modify the Operational Semantics 21

Modify the Operational Semantics Branch instructions require green and blue computations to agree 22

Syntactic Semantic FT Summary ü ü Machine syntax ü ü Operational semantics ü ü Typechecker ü ü Type systems ü ü Definition of “safe” to include fault states • Safety (colors, no faults) ü Model faults Safety in the presence of faults 23

Outline ü Introduction ü Organization of the List-Machine framework ü Extend the List Machine with fault tolerance • Semantic and syntactic methods in large systems 24

How Semantic and Syntactic Methods Scale Princeton Foundational Proof-Carrying Code (FPCC) Vs. Carnegie Mellon Con. Cert project FPCC : : Semantic Con. Cert : : Syntactic 25

Common Traits • Include a TAL for ML compiled to machine code • Goal: guarantee a memory property for untrusted code • Written in Twelf • Industrial-strength TALs • Large systems 26

Composition Checker – theorem checker for FPCC and a metatheorem checker for Con. Cert Machine – SPARC or x 86 definitions Trusted Computing Base Logic – example: definition of modular arithmatic Theorems – statement of the safety property Proof T + L + M << P 27

Token count of TCB components 400000 350000 Checker Runtime 300000 Policy 250000 Machine Definition Axioms 200000 150000 100000 50000 0 FPCC Con. Cert 28

Token count of TCB components 30000 Checker 25000 Runtime Policy 20000 Machine Definition Axioms 15000 10000 5000 0 FPCC Con. Cert The TCBs are equivalent in size except for the Checker 29

Interface Safety Requires • updating the policy • moving the type system from Proof to Theorem – now part of the TCB Should the type system be semantic or syntactic? 30

Scaling Law Semantic: new definition per type constructor Syntactic: new definition per expression constructor Toy systems have few expression constructors… 31

Real systems have more expression constructors than type constructors. semantic methods require fewer definitions Is the average type definition larger than the average typing rule? 32

In toy systems, typing rules are simple. . . |- stmt_prim_lbladd_ADD_imm: judge_stmt (e_prim A (p_lbladd V 1 (val_diff L 0 Lab I 2))) Prog L CCEnv AENV KL Ps Phi L' CCEnv AENV KL Ps' Phi' <regbind A At Prog <- targetreg At Ar <regbind_val Prog V 1 Vt <- realreg Vt Vr <diff_value Prog (val_diff L 0 Lab I 2) Vc <imm 13 Vc (c Vimm 13) <value. Ty Prog KL Phi V 1 (offset I 1 (int pi= (addr Lab))) <value. Ty Prog KL Phi (val_diff L 0 Lab I 2) (offset I 2 (diff L 0 Lab)) <check_lbladd_offset I 1 I 2 <num_add I 1 I 2 I 1+I 2 <venv_add Prog A (offset I 1+I 2 (int pi= (addr L 0))) Phi' <decode_list L L' Ps Ps' (instr_ADD Vr (inject_imode Vimm 13) Ar) =. . . 33

How does this balance in FPCC & Con. Cert? 35000 30000 25000 20000 15000 10000 5000 0 Size of Type System Specification Semantic FPCC Syntactic FPCC Con. Cert (XTALT) Con. Cert (TALT) • FPCC’s semantic definitions are half the size of syntactic definitions for FPCC • This will become even more pronounced according to the scaling law if the compiler wishes to generate more instructions. 34

Conclusion ü Introduction ü Organization of the List-Machine framework ü Extend the List Machine with fault tolerance ü Semantic and syntactic methods in large systems 35

Appendix 36

Modified Typing Rules 37

Modified Operational Semantics w = (n, ρ, a) w = (n, ρ, a, ρ’, κ) • ρ’ – FT register store • κ – color store (and equivalent for the syntactic system) 38

Modified Semantic Type System 39

List-Machine Benchmark 2. 0 • Easily extended • Facilitates small scale comparisons between many proof methods (semantic and syntactic). 40

Princeton’s Foundational Proof Carrying Code (FPCC) vs Carnegie Mellon’s Con. Cert • Compare how type systems scale between semantic and syntactic proof methods 41

Modules 42

Type System Specification Typechecker Soundness Proof check(Π, Ψ) = true → Π ⊢blocksΨ Type System Π⊢blocks Ψ → safe Ψ Typechecking Algorithm check(Π, Ψ) = true 43

Type System Specification • type operators • definitions of typing rules • statement of safety • Π⊢blocks Ψ → safe Ψ Typechecker Soundness Proof check(Π, Ψ) = true → Π ⊢blocksΨ Type System Proves: Π⊢blocks Ψ → safe Ψ Typechecking Algorithm check(Π, Ψ) = true 44

Modules 45