AVY ArieVeYakir Emilia Katz Intel 051117 Many slides

AVY “Arie-Ve-Yakir” Emilia Katz Intel, 05/11/17 (*)Many slides are borrowed from Yakir Vizel and Arie Gurfinkel

Agenda • All roads lead to Rome • ISC: Interpolation-Sequence based MC [Yakir Vizel, Orna Grumberg: Interpolation-sequence based model checking. FMCAD 2009: 1 -8] • PDR: Property-Directed Reachability [many papers; current presentation mostly based on G. Cabodi, P. Camurati, A. Mishchenko, M. Palena, P. Pasini: SAT solver management strategies in IC 3: an experimental approach. Formal Methods in System Design 50(1): 39 -74 (2017)] • AVY: combine and conquer [Yakir Vizel, Arie Gurfinkel: Interpolating Property Directed Reachability. CAV 2014: 260276]

Safety Property Model Checking • Given: a transition system M = (V, Init, Tr, Bad) • Is M “safe”? (Is Bad reachable in M? ) • M is UNSAFE if and only if there exists a number N s. t. • M is SAFE if and only if there exists a safe inductive invariant Inv s. t. Inductive Safe

Preliminaries: BMC • Bounded Model Checking • Does the system have a counterexample of length k? …… INIT R 1 BAD R 2 Rk-1 INIT(V 0) ∧Tr(V 0, V 1) ∧…∧Tr(Vk-1, Vk)∧Bad(Vk)

Preliminaries: Inductive Trace An inductive trace of a transition system M = (V, Init, Tr, Bad) is a sequence of formulas [F 0, …, FN] such that § Init → F 0 § ∀ 0 ≤ i < N , Fi(V) ∧ Tr (V, V’) → Fi+1 (V’) A trace is safe iff ∀ 0 ≤ i ≤ N , Fi → ¬Bad A trace is closed iff ∃ 1 ≤ i ≤ N, Fi → (F 0 ∨ … ∨Fi-1) A trace is monotone iff A trace is clausal iff ∀ 0 ≤ i < N , Fi → Fi +1 every Fi is CNF al: d o G nde u o Unb MC

Preliminaries: Interpolation-Based MC • Given the following BMC formula B A INIT(V 0) ∧ Tr(V 0 , V 1) ∧ Tr(V 1 , V 2) ∧ … ∧ Tr(Vk-1 , Vk) ∧ Bad(Vk) I A⇒I I∧B ≡F I is over common variables of A, B

Preliminaries: Interpolation-Based MC • Iteration of over-approximated forward reachability for “k-safety” B 1 A 1 INIT(V 0) ∧ Tr(V 0 , V 1) ∧ Tr(V 1 , V 2) ∧ … ∧ Tr(Vk-1 , Vk) ∧ (⋁kj=1 Bad(Vj)) B 2 A 2 I 1 k(V 0) ∧ Tr(V 0 , V 1) ∧ Tr(V 1 , V 2) ∧ … ∧ Tr(Vk-1 , Vk) ∧ (⋁kj=1 Bad(Vj)) Bn Ink(V 0) ∧ Tr(V 0 , V 1) ∧ Tr(V 1 , V 2) ∧ … ∧ Tr(Vk-1 , Vk) ∧ (⋁kj=1 Bad(Vj)) I 2 k (V 2) … … An I 1 k(V 1) Ink (Vn)

Interpolation-Sequence • Different partitioning of the same BMC formula A 1 A 3 A 2 Ak+1 Ak INIT(V 0) ∧ Tr(V 0 , V 1) ∧ Tr(V 1 , V 2) ∧ Tr(V 2 , V 3) ∧ … ∧ Tr(Vk-1 , Vk) ∧ Bad(Vk) I 1 I 0 = T , Ik+1 = F Ij-1 ∧ Aj → Ij I 2 I 3 Ik-1 Inductive trace <I 0, I 1, … , Ik-1> Ij is over Vj (= Var(A 1∧ …∧Aj ) ∩ Var(Aj+1∧ …∧Ak+1) ∀1⩽j<k+1 Ij = Itp(A 1 ∧ …∧ Aj , Aj+1 ∧ …∧ Ak+1) Ik

Interpolation-Sequence-Based MC N=1 Imc. Mk. Safe BMCN SAT CEX UNSAT Seq. Itp N: =N+1 trace F = [F 0, …, FN] No Is F closed Yes SAFE

Interpolation-Sequence-Based MC Take: I 0 = T , Ik+1 = F Goal: Cycle i: over-approximate Ri, refine previous approximation of R 1…Ri-1 (1) Ij → q Example: i=1 (2) Ij-1 ∧ Aj → Ij (3) Rj → Ij II 1, 1 1 I 1, 2 I 2, 2

Interpolation Sequences as Traces Take: I 0 = T , Ik+1 = F Goal: (1) Ij → q (2) Ij-1 ∧ Aj → Ij INIT R 1 R 2 R 3 (3) Rj → Ij BAD ¬q INIT II 11 I 3 II 2 2 II 1, 2 1, 1 1, 3 II 2, 3 2, 2 I 3, 3

PDR idea Incrementally construct a forward reachability sequence [F 0, …, FN] with: 1. F 0=INIT monotone 2. Fi → Fi+1 3. ∀ 0 ≤ i < N , Fi(V) ∧ Tr (V, V’) → Fi+1 (V’) 4. ∀ 0 ≤ i < N , Fi → ¬Bad safe Meaning: § F 0 represents the initial states (1) § Fi over-approximates states reachable in i steps or less (2+3)

PDR - overview F = [Init, ¬ Bad] monotone, safe, clausal PDR Invariants Fi ⇒ ¬ Bad Init ⇒ Fi+1 F’i+1 Fi ∧ Tr ⇒ PDR trace: Mk. Safe CEX G = [G 0, …, GN] Propagate c from Fj to Fj+1 iff Push Fj ∧ Tr ⇒ c’ F = [F 0, …, FN]; FN = ¬ Bad(*) (*)This is for simplicity. Initialzation of FN is actually done differently (by Push from FN-1) PDR Push: F = [F 0, …, FN] No ∃ i. Fi = Fi+1 Yes SAFE

PDR– Initialization 1. 2. 3. Start by checking the following two formulae 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F=… Mk. Safe Push No § INIT ˄ ¬p, and Yes ∃ i. … § INIT ˄ Tr ˄ ¬p’ § If Un. SAT then INIT → p and INIT ˄ Tr → p’ R 0 R 1 p CEX SAF E

PDR – Initialization 1. 2. 3. Start by checking the following two formulae 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F=… Mk. Safe Push No Yes ∃ i. … § INIT ˄ ¬p, and § INIT ˄ Tr ˄ ¬p’ § If Un. SAT then INIT → p and INIT ˄ Tr → p’ § Initialize F 0 = INIT and F 1 = p § [F 0, F 1] F 0 F 1=p CEX

1. 2. 3. Pdr. Mk. Safe – Try to expand the trace F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p 4. Check whether F 1 ˄ Tr → p’ F = … Mk. Safe Push No § If not, there exists a state s such that F 1 ˄ Tr ˄ ¬p’ is SAT s F 0 F 1=p Yes ∃ i. … CEX

Pdr. Mk. Safe – Try to expand the trace 1. 2. 3. A state s, that can reach ¬p, is found in F 1 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe Push No Yes ∃ i. … § Find out whether it is reachable § But since it is in F 1, only need to check if it is reachable from F 0 § F 0 ˄ Tr ˄ s’ s F 0 F 1=p CEX

Pdr. Mk. Safe – Try to expand the trace 1. 2. 3. A state s, that can reach ¬p, is found in F 1 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe Push No Yes ∃ i. … § Find out whether it is reachable § But since it is in F 1, only need to check if it is reachable from F 0 § F 0 ˄ Tr ˄ s’ § If it is SAT – a counterexample exists § s has a predecessor in INIT s F 0 F 1=p CEX

Pdr. Mk. Safe – Try to expand the trace 1. 2. 3. A state s, that can reach ¬p, is found in F 1 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe Push No Yes ∃ i. … § Find out whether it is reachable § But since it is in F 1, only need to check if it is reachable from F 0 § F 0 ˄ Tr ˄ s’ § If it is Un. SAT – s can be removed from F 1 § It is not reachable s F 0 F 1=p CEX

1. 2. 3. Pdr. Mk. Safe – Try to expand the trace 4. A state s in F 1 can reach ¬p, and is not reachable F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p Push Yes ∃ i. … § s cannot be in F 0 § s cannot be reached from F 0 – meaning, F 0 ˄ Tr → ¬s’ § Conclusion: (F 0 ˄ ¬s ) ˄ Tr → ¬s’ § F 1 = F 1 ˄ ¬s s F 0 Mk. Safe No What else do we know? § Removing s from F 1 F = … F 1=p CEX

Pdr. Mk. Safe – Try to expand the trace 1. 2. 3. 4. After removing s, check whether F 1 ˄ Tr → p’ and iterate this process F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p When F 1 ˄ Tr → p’ holds, expand with F 2=p § [F 0, F 1, F 2] F 0 F 1 F 2=p F = … Mk. Safe Push No Yes ∃ i. … CEX

Pdr. Mk. Safe – k-th iteration 1. 2. 3. The trace is [F 0, F 1, …, Fk] 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe Push No Yes ∃ i. … Check whether Fk ˄ Tr → p’ § If not, there exists a state s… s F 0 F 1 … Fk p CEX

Pdr. Mk. Safe – k-th iteration 1. 2. 3. Check whether s is reachable from Fk-1 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe Push No Yes ∃ i. … § Fk-1 ˄ Tr ˄ s’ § If it is, there exists a state t… s F 0 F 1 … Fk p CEX

Pdr. Mk. Safe – k-th iteration 1. 2. 3. The k-th iteration performs a backward search… 4. F = … F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p Mk. Safe Push No Yes ∃ i. … F 0 F 1 r Fk-2 Fk-1 Fk p CEX

Pdr. Mk. Safe – k-th iteration 1. 2. 3. Let’s assume r cannot be reached from Fk-3 4. F = … F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p Mk. Safe Push No § (Fk-3 ˄ ¬r ) ˄ Tr → ¬r’ Yes ∃ i. … § r is removed from Fk-2 § Try and find a different predecessor to t F 0 F 1 r t Fk-2 s Fk-1 Fk p CEX

Pdr. Mk. Safe – k-th iteration 1. 2. 3. Let’s assume r cannot be reached from Fk-3 4. F = … F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p Mk. Safe Push Yes No § (Fk-3 ˄ ¬r ) ˄ Tr → ¬r’ ∃ i. … § r is removed from Fk-2 § Try and find a different predecessor to t q F 0 F 1 t Fk-2 s Fk-1 Fk p CEX

Inductiveness 1. 2. 3. What do we know when a state s in Fj is not reachable 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe Push No § It cannot be in Fj-1 Yes ∃ i. … § Otherwise we would have found it earlier § And it is not needed in Fj We therefore know that (Fj -1 ˄ ¬s ) ˄ Tr → ¬s’ § What does it look like? ¬s is inductive relative to Fj-1 INIT → ¬s CEX

Inductive Generalization 1. 2. 3. Let s be blocked in Fj 4. F = … F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p Mk. Safe CEX Push Yes No ∃ i. … s F 0 F 1 Fj-1 Fj p

Inductive Generalization 1. 2. 3. Try and generalize ¬s 4. F = … F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p Mk. Safe CEX Push Yes No ∃ i. … § We want to learn a stronger fact § What does it mean stronger? § c → ¬s s F 0 F 1 Fj-1 Fj p c

Inductive Generalization 1. 2. 3. Try and generalize ¬s 4. F = … F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p Mk. Safe CEX Push Yes No ∃ i. … § We want to learn a stronger fact § What does it mean stronger? § c → ¬s c should be inductive relative to Fj-1 s § INIT→ c § (Fj-1 ˄ c ) ˄ Tr → c’ F 0 F 1 Fj-1 Fj p c

Inductive Generalization 1. 2. 3. Try and generalize ¬s 4. § We want to learn a stronger fact F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe CEX Push No Yes ∃ i. … Pushing c SAF E § What does it mean stronger? § c → ¬s c should be inductive relative to Fj-1 s § INIT→ c § (Fj-1 ˄ c ) ˄ Tr → c’ F 0 F 1 Fj-1 Fj p Fj = Fj ˄ c § We remove a set of states from Fj c

Inductive Generalization 1. 2. 3. Try and generalize ¬s 4. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe CEX Push No § We want to learn a stronger fact Yes ∃ i. … SAF E § What does it mean stronger? § c → ¬s c should be inductive relative to Fj-1 s § INIT→ c § (Fj-1 ˄ c ) ˄ Tr → c’ F 0 F 1 Fj-1 Fj p Fj = Fj ˄ c § We remove a set of states from Fj c

Pdr. Push: Pushing for Termination 1. 2. 3. Termination condition: closed trace 4. § Fixpoint reached with Fj+1 → Fj § Due to monotonicity… § Syntactic check for fixpoint: § Fj and Fj+1 contain the same clauses Note duality of inclusion: § As sets of states: § As sets of clauses: F 0 ⊇ F 1 ⊇ … ⊇ FN F 0 ⊆ F 1 ⊆ … ⊆ FN F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 Fj → p F = … Mk. Safe CEX Push No Yes ∃ i. … SAF E

1. 2. 3. F 0=INIT Fj → Fj+1 Fj ˄ Tr → F’j+1 4. Fj → p Try and “force” two subsequent trace elements to be equal Pdr. Push: Pushing for Termination § Move clauses from Fj to Fj+1 § Clause c in Fj is “pushable” when: § Fj ˄ Tr → c’ is valid § Idea: create inductive property F § F ˄ Tr → F’ § Example: § Assume: F=c 1 ˄ c 2 ˄ c 3 ; ∀1⩽i ⩽ 3. Init → ci § If we can show: ∀1⩽i ⩽ 3. F ˄ Tr → ci’ F is inductive F = … Mk. Safe CEX Push No Yes ∃ i. … SAF E

PDR - overview F = [Init, ¬ Bad] monotone, safe, clausal PDR Invariants Fi ⇒ ¬ Bad Init ⇒ Fi+1 F’i+1 Fi ∧ Tr ⇒ PDR trace: Mk. Safe CEX G = [G 0, …, GN] Propagate c from Fj to Fj+1 iff Push Fj ∧ Tr ⇒ c’ F = [F 0, …, FN]; FN = ¬ Bad(*) (*)Initialzation of F N can be done differently (e. g. , by Push from FN-1) PDR Push: F = [F 0, …, FN] No ∃ i. Fi = Fi+1 Yes SAFE

AVY - Intuition • Combine Sequential Interpolation and PDR • Seq. Int is good at extending the trace • Easy to find an interpolant • Each frame starts from closer over-approximation that of PDR • PDR is good at convergence: • Makes the trace monotone • Clause propagation (“pushing”) enables faster arrival to a fixed point

Avy: Interpolating PDR N=1 § Bounded verification with BMC G = [G 0, …, GN] BMCN SAT CEX § Locally convert (and strengthen) to PDR trace UNSAT § Re-use old trace G in new BMC step Seq. Itp N: =N+1 § Global trace using sequence interpolation trace F = [F 0, …, FN] § Compute strengthening of old trace G by interpolation Mk. Pdr. Trace PDR trace G = [G 0, …, GN] No ∃ i. Gi = Gi+1 Yes SAFE

AVY – Implementation Input: Transition system M = (Init, Tr, Bad) Data: A trace F = [F 0, …, FN ] global trace F 0 : = Init ; N : = 0 reuse prev. repeat frame BMCN : = ⋀i=0 N-1 [Fi (vi) ∧ Tr(vi, vi+1)] i ∧ [FN(v. N) ∧ Bad(v. N)] N if is. Sat(BMCN) then stronger, monotone return UNSAFE; sub-trace I 1, …, IN : = Seq. Itp(BMCN) stronger, monotone [ _; F 1] : = Pdr. Mk. Safe([Init, F 1], ¬(Init ∨ I 1)) current sub-trace for i = 2 to N do [_, _, Fi] : = Pdr. Mk. Safe([Init, Fi-1, Fi ], ¬(Fi-1 ∨ Ii)) strengthen future trace; F 0, …, FN : = Pdr. Push([F 0, …, FN ]) end speed up convergence // F 0, …, FN is a safe trace syntactic termination if ∃ 0 ≤ i < N · Fi = Fi+1 then return SAFE; step = 1 N : = N + 1 end

AVY – Usage of Pdr. Mk. Safe Goal: strengthen each Fi ; make monotone Give to PDR: [Init, Fi-1, Fi ], ¬(Fi-1 ∨ Ii)), - not safe, not monotone Correctness intuition: 1. Can make safe (“New Bad” is good): § § § For i=1, “New Bad” = ¬(Init ∨ I 1) § R 1 ∧ ¬ (Init ∨ I 1) = UNSAT, as [Init, I 1] is a trace We can use New. Bad to refine F 1 For i=2, “New Bad” = ¬(F 1 ∨ I 2) § From above, F 1(u) → Init ∨ I 1 § Both [I 0, I 1 , I 2], and [Init, F 1] are traces § I 1(u) ∧ Tr(u, v)→ I 2(v) § Init(u) ∧ Tr(u, v)→ F 1(v) § (F 1(u) ∧ Tr(u, v)) → (F 1(v) ∨ I 2(v)) Similar reasoning can be used for i>2 RF I 011 R 1 RF 01 R 1 I 2 p p R 2

AVY – Usage of Pdr. Mk. Safe Goal: strengthen each Fi ; make monotone Give to PDR: [Init, Fi-1, Fi ], ¬(Fi-1 ∨ Ii)), get safe monotone trace Correctness intuition: 2. Combination ([F 0, F 1 , …, FN]) of the above traces is a safe monotone trace: § § § For i=1, § [F 0 (= Init), old_F 1] and [Init, F 1] are safe monotone traces [Init ∧ F 0, old_F 1 ∧ F 1] is a safe monotone trace § F 1 is a refinement of old_F 1 ∧ F 1 = F 1 [F 0, F 1] is a safe monotone trace For i=2, § Bad = ¬(F 1 ∨ I 2); F 1 ∧ Tr → F 1 ∨ I 2 we don’t need to remove states before F 1 while blocking states of F 2 § Both [Init (=F 0), F 1 ], and [Init, F 1 , F 2] are safe monotone traces [F 0, F 1 , F 2] is a safe monotone trace Similar reasoning can be used for i>2

Extending a Trace by PDR Pdr. Mk. Safe ¬ (F 1 ∨ I 2) Observations: [Init, F 1, F 2] is a PDR trace F 2 is stronger than F 1 ∨ I 2 F 1 after is stronger than F 1 before!!! Frame F 0 Frame F 1 Frame F 2

Avy: Interpolating PDR - summary N=1 § Bounded verification with BMC G = [G 0, …, GN] BMCN SAT CEX § Locally convert (and strengthen) to PDR trace UNSAT § Re-use old trace G in new BMC step Seq. Itp N: =N+1 § Global trace using sequence interpolation trace F = [F 0, …, FN] § Compute strengthening of old trace G by interpolation Mk. Pdr. Trace PDR trace G = [G 0, …, GN] No ∃ i. Gi = Gi+1 Yes SAFE

Experimental Results Started with an implementation based on ABC § slightly modified PDR engine with external API § added Sequence Interpolation SAT solving with Mini. SAT and Glucose § search for a good proof with one solver § re-solve to compute interpolants Performs differently from PDR § virtual best is much better than either one in isolation Status AVY PDR SEQ Virtual Best SAFE 76 72 62 112 UNSAFE 24 15 26 29

Thank you!