A Theory of Mutations with Applications to Vacuity

A Theory of Mutations with Applications to Vacuity, Coverage, and Fault Tolerance Orna Kupferman 1 Wenchao Li 2 Sanjit A. Seshia 2 1 Hebrew University 2 UC Berkeley FMCAD 2008 1

FMCAD 2008 2

This system is correct even under faults (e. g. flips in latches) Why? Convince me. It satisfies its specification under these faults. Doesn’t this mean the specification coverage is low? Adam So is my specification not good enough or is my system fault-tolerant? Bob Need fault-tolerance! But also need to certify it! FMCAD 2008 3

Problem o Current mutation-based metrics are inadequate to reason about specification coverage for fault-tolerant circuits in model checking. FMCAD 2008 4

Preliminaries o Coverage n o o Vacuity Introduce ∆ to an implementation I and check I’ ² S. n Introduce ∆ to a specification S and check I ² S’. Fault Tolerance n I with fault f still satisfies S. All three involve introducing mutations in the verification process! FMCAD 2008 5

Contributions A theory of mutations: o formally ties together coverage and vacuity in model checking; o enables reasoning coverage for fault-tolerant circuits. FMCAD 2008 6

Agenda o Related Work n n o A Theory of Mutations n n o o Coverage Vacuity Coverage and Vacuity are dual Aggressiveness amongst mutations Applications Conclusion FMCAD 2008 7

Coverage o o Is my specification complete? Coverage metrics for model checking [HKHZ 99; KGG 99; CKV 01, 03] path state FSM Coverage FMCAD 2008 8

Coverage o o o Functional Coverage in BMC [GKD 07] Detect “forgotten cases” [Claessen 07] Coverage for fault-tolerant systems [FPFRT 03, DBBDCMF 05] n Single stuck-at fault model FMCAD 2008 9

Vacuity o o Is my specification satisfied trivially? Vacuity detection [KV 99, 03; BBER 01; AFFGP 03; CG 04; BFGKM 05; BK 08] G (req → F grant) G (req → false) Replace a sub-formulae in the most challenging way. Trivially true in a system where req is never sent. FMCAD 2008 10

Agenda o Related Work n n o A Theory of Mutations n n o o Coverage Vacuity Coverage and Vacuity are dual Aggressiveness amongst mutations Applications Conclusion FMCAD 2008 11

Examples of Mutations o o Can mutate inputs, outputs, or latches 1001 Stuck-at 1000 o 1000 Modifies behaviors Restricting a signal to a value 1000 o 1001 Removes behaviors Freeing (abstracting) a signal old 1000 1001 new 100 X FMCAD 2008 Adds behaviors 12

A Theory of Mutations o Properties: n n n o Invertability: (Cμ)ν = C Monotonicity: I ² S → Iμ ² Sμ Duality Interesting Mutations: n n n Conditional stuck-at Conditional add/remove transitions Permuting events FMCAD 2008 13

Duality Iμ ² S ↔ I ² S ν low coverage vacuity , where ν and μ are dual mutations. FMCAD 2008 14

z Circuit with input = {z}, control signals = {x, y}, output = {x}, described by the state representation on the right. xy x S simulates I’ and S’ simulates I 0, 1 S 01 0 add behavior remove behavior 0, 1 I’ 0, 1 01 0 00 0 0 1 11 1 10 1 0, 1 11 1 0, 1 S’ 0 01 0 FMCAD 2008 1 00 0 15

Aggressiveness o o Mutation is more aggressive than if applying makes it harder for the design to satisfy its specification. I ² S → I ² S ≥imp or ≥spec I ² S → I ² S FMCAD 2008 16

Some Aggressive Orders o o o Free(x) ≥ k-SEU(x) Free(x) ≥ Stuck_at_0(x) Free(x) ≥ Flip(x) Delay_k+1 ≥ Delay_k k-SEU(x) ≥ m-SEU(x) ≥ for k ≥ m More interesting ones can be found in the paper. FMCAD 2008 17

Coverage for Fault-tolerance o For a fault-tolerant system I and a set of mutations { j} such that n o I j ² S for all 1≤j≤k. The fault-tolerant system loosely satisfies S if there is a mutation such that n j ≤imp for all 1≤j≤k; n I ² S. FMCAD 2008 18

Agenda o Related Work n n o A Theory of Mutations n n o o Coverage Vacuity Coverage and Vacuity are dual Aggressiveness amongst mutations Applications Conclusion FMCAD 2008 19

Applications o o o Useful vacuity information can be obtained for free from coverage checks. Analyze coverage for fault-tolerant systems. Improving specifications n n Catch bugs Strengthen environmental assumptions FMCAD 2008 20

Vacuity from Coverage o o S: G (sp[2. . 0] = 3’b 110 → X (sp[2. . 0] = 3’b 111) In our experiment, applying the “Flip(x)” mutation to sp[0] still satisfies S. S’: G (sp[2. . 0] = 3’b 110 → X (sp[2. . 0] = 3’b 110) S & S’ → G ¬(sp[2. . 0] = 3’b 110) FMCAD 2008 21

Certifying Fault-Tolerance 1 -SEU System behaviors 2 -SEU Original low-coverage spec. FMCAD 2008 High-coverage spec. certifies system’s target resilience 22

Experiments VIS benchmarks, results obtained with Cadence SMV model checker FMCAD 2008 23

Improving Specifications o Chip Multiprocessor Router [Peh 01] Simplied model S: G (ξ → X ¬(grant = 2’b 11) S’: G (ξ → X (grant = 2’b 10) o However, the process still requires some user assistance. FMCAD 2008 24

Conclusion o A theory of mutations that n n o o Unifies coverage and vacuity Can be used to certify the correctness of fault-tolerant circuits A new technique to tighten specifications The ideas here can be applied to other verification techniques. FMCAD 2008 25

Q&A Thank you! FMCAD 2008 26

References FMCAD 2008 27