Asserting and Checking Determinism for Parallel Programs Jacob
Asserting and Checking Determinism for Parallel Programs Jacob Burnim Koushik Sen* EECS, UC Berkeley
Motivation § Parallel programming is difficult § Culprit: Non-determinism • Interleaving of parallel threads • But required to harness parallelism § Sequential programs produce deterministic results § To make parallel programming easy • We want determinism • Same input => semantically same output.
Determinism Efforts § Language design [DPJ, X 10, Yada] • • • Deterministic by design: Types and annotations Constrains programming May need non-determinism for better performance § Deterministic runtime [DMP, Kendo] § Race detection • • § Determinism Checker [Single. Track, Torrellas et al. ] • § Absence of data races does not imply determinism Data races could be benign and could help in improving performance Dynamic determinism checker. Determinism with respect to abstraction [Rinard et al. ]
Our Goal § Provide a framework that can express relevant determinism directly and easily • Separate from functional correctness specification • Allow data races and healthy nondeterminism • Language independent • Useful for QA
Our Goal § How to specify correctness of parallelism? Implicit: Explicit: No sources of non-determinism (no data races) Full functional correctness. Determinism specification: A sweet spot? • Lightweight, but precise.
Outline § Motivation § Deterministic Specification [FSE’ 09] § Checking: Active Testing § Experimental Evaluation § Future Work + Conclusions
Deterministic Specification // Parallel fractal render mandelbrot(params, img); § Goal: Specify deterministic behavior. • Same initial parameters => same image. • Non-determinism is internal.
Deterministic Specification deterministic { // Parallel fractal render mandelbrot(params, img); } § Program: , Initial State: § Specifies: Two runs from same initial program state have same result state for any pair of schedules , Schedules:
Deterministic Specification double A[][], b[], x[]; . . . deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); } § Too restrictive – different schedules may give slightly different floating-point results.
Deterministic Specification set t = new Red. Black. Tree. Set(); deterministic { t. add(3) || t. add(5); } § Too restrictive – internal structure of set may differ depending on order of adds.
Deterministic Specification deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); } § Too restrictive – search can correctly return any tree with optimal cost.
Semantic Determinism § Too strict to require every interleaving to give exact same program state: deterministic { P }
Semantic Determinism § Too strict to require every interleaving to give exact same program state: deterministic { P } Predicate! Should be user-defined.
Semantic Determinism § Too strict to require every interleaving to give exact same program state: deterministic { P } assert Post(s 1, s 1’) § Specifies: Final states are equivalent.
Semantic Determinism double A[][], b[], x[]; . . . deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); } assert (|x – x’| < ε) “Bridge” predicate
Semantic Determinism set t = new Red. Black. Tree. Set(); deterministic { t. add(3) || t. add(5); } assert (t. equals(t’)) § Resulting sets are semantically equal.
Semantic Determinism deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); } assert (t. cost == t’. cost())
Preconditions for Determinism set t = … deterministic { t. add(3) || t. add(5); } assert (t. equals(t’)) … deterministic { t. add(4) || t. add(6); } assert (t. equals(t’)) § Too strict – initial states must be identical • Not compositional.
Preconditions for Determinism § Too strict to require identical initial states: deterministic { P } assert Post(s 1, s 1’)
Preconditions for Determinism § Too strict to require identical initial states: deterministic assume (s 0 = s 0’) { P } assert Post(s 1, s 1’)
Preconditions for Determinism § Too strict to require identical initial states: deterministic assume (s 0 = s 0’) { P } assert Post(s 1, s 1’) Predicate! Should be user-defined.
Preconditions for Determinism § Too strict to require identical initial states: deterministic assume Pre(s 0, s 0’) { P } assert Post(s 1, s 1’) § Specifies:
Bridge predicates/assertions deterministic assume Pre(s 0, s 0’) { P } assert Post(s 1, s 1’) “Bridge” predicate “Bridge” assertion
Preconditions for Determinism set t =. . . deterministic assume (t. equals(t’) { t. add(4) || t. add(6); } assert (t. equals(t’)) § Specifies: Semantically equal sets yield semantically equal sets.
Advantage § Separately check parallelism and functional correctness. • Show parallelism is outwardly deterministic. • Reason about correctness sequentially. • Decompose correctness proof! § Example: • Write Cilk program and prove (or test) sequential correctness. • Add parallelism, answers should not change
Determinism vs. Atomicity § Internal vs. external parallelism/non-determinism • Complementary notions Deterministic “Closed program” Atomic “Open program”
Other Testing Use sequential program as spec Regression testing deterministic Pre(s 0, s 0’) { if (*) { Sequential. Pgm; } else { Pgm. Version 1; } else { Parallel. Pgm; } } assert Post(s, s’); Pgm. Version 2; } } assert Post(s, s’);
Outline § Motivation § Deterministic Specification [FSE’ 09] § Checking: Active Testing [PLDI 08, 09, FSE 08, CAV 09] § Experimental Evaluation § Future Work + Conclusions
Checking: Active Testing Predicting and Exploring “interesting” Schedules
Active Testing: Predict and Test Potential Bugs § Predict potential bugs: • Data races: Eraser or lockset based • Atomicity violations: cycle in transactions and happens-before relation • Deadlocks: cycle in resource acquisition graph • Memory model bugs: cycle in happensbefore relation § Test schedules to create those bugs [ASE 07, PLDI 08, FSE 08, PLDI 09, FSE 09, CAV 09]
Active Testing Cartoon: Phase I Potential Collision 2 1 1 3 2 31
Active Testing Cartoon: Phase II 2 1 1 3 2 32
Outline § Motivation § Deterministic Specification § Checking: Active Testing § Experimental Evaluation § Future Work + Conclusions
Ease of Asserting Determinism § Implemented a deterministic assertion library for Java. § Manually added deterministic assertions to 13 Java benchmarks with 200 – 4 k Lo. C § Typically 5 -10 minutes per benchmark • Functional correctness very difficult.
Ease of Use: Example Deterministic. open(); Predicate eq = new Equals(); Deterministic. assume(width, eq); … (9 parameters total) … Deterministic. assume(gamma, eq); // Compute fractal in threads int matrix[][] = …; Deterministic. assert(matrix, eq); Deterministic. close();
Effectiveness in Finding Bugs § 13 Java benchmarks of 200 – 4 k Lo. C § Ran benchmarks on ~100 schedules • Schedules with data races and other “interesting” interleavings (active testing) § For every pair of executions of deterministic Pre { P } Post: check that:
Experiments: Java Grande Forum Benchmark Lo. C Data Races Found | Errors High-Level Races Found | Errors sor 300 2 0 0 0 moldyn 1. 3 k 2 0 0 0 lufact 1. 5 k 1 0 0 0 raytracer 1. 9 k 3 1 0 0 montecarlo 3. 6 k 1 0 2 0
Experiments: Parallel Java Lib Benchmark Lo. C pi Data Races Found | Errors High-Level Races Found | Errors 150 9 0 1+ 1 keysearch 3 200 3 0 0+ 0 mandelbrot 250 9 0 0+ 0 phylogeny 4. 4 k 4 0 0+ 0 6 0 2 0 tsp* 700
Experimental Evaluation § Across 13 benchmarks: § Found 40 data races. • 1 violates deterministic assertions.
Experimental Evaluation § Across 13 benchmarks: § Found 40 data races. • 1 violates deterministic assertions. § Found many interesting interleavings (non-atomic methods, lock races, etc. ) • 1 violates deterministic assertions.
Determinism Violation deterministic { // N trials in parallel. foreach (n = 0; n < N; n++) { x = Random. next. Double(); y = Random. next. Double(); … } } assert (|pi - pi’| < 1 e-10) § Pair of calls to next. Double() must be atomic.
Outline § Motivation § Deterministic Specification § Checking: Active Testing § Experimental Evaluation § Future Work + Conclusions
How to verify: Essential Rule deterministic assume(Φ 1) A 11; A 12; A 13|| A 21; A 22; A 23|| A 31; A 32; A 33 } assert (Φ 2)
How to verify: Essential Rule deterministic assume(Φ 1) A 11; A 12; A 13|| A 21; A 22; A 23|| A 31; A 32; A 33 } assert (Φ 2) Find an invariant Φ and prove that for all i, j, l, m deterministic assume(Φ) Aij|| Alm } assert (Φ) and Φ 1 → Φ and Φ → Φ 2
Prove for the following deterministic assume(true) { parallel while (!wq. is_empty()) { [work = wq. get(); ] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best, cost(s)); ] } else { [wq. add_all(split(work)); ] } } } assert(best=best’)
Decompose the proof Let us prove that P ≈ S That is prove that deterministic assume(Pre(S 0, S’ 0)) { if (*) P else S } assert (Post(S, S’))
Decompose the proof Let us prove that P ≈ S Create a non deterministic sequential program NS [Galois] P ≡ NS ≈ S
Decompose the proof Let us prove that P ≈ S Create a non deterministic sequential program NS P ≡ NS ≈ S deterministic assume(S 0=S’ 0) { if (*) P else NS } assert (S=S’) deterministic assume(Pre(S 0, S’ 0)) { if (*) NS else S } assert (Post(S, S’))
deterministic assume(true) { How to verify? parallel while (!wq. is_empty()) { Not deterministic sequential program while (!wq. is_empty()) { [work = wq. get(); ] [work = wq. nondet_get(); ] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best, cost(s)); ] } else { [wq. add_all(split(work)); ] } } assert(best=best’) }
deterministic assume(true) { How to verify? parallel while (!wq. is_empty()) { Not deterministic sequential program while (!wq. is_empty()) { [work = wq. get(); ] [work = wq. nondet_get(); ] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); Barrier() s = find_best_solution(work); [best = min(best, cost(s)); ] } else { [wq. add_all(split(work)); ] } } assert(best=best’) }
deterministic assume(true) { How to verify? parallel while (!wq. is_empty()) { Not deterministic sequential program while (!wq. is_empty()) { [work = wq. get(); ] [work = wq. nondet_get(); ] if (bound(work) >= best) if (bound(work) >= best && non_det) continue; if (size(work) <= threshold) { s = find_best_solution(work); Barrier() s = find_best_solution(work); [best = min(best, cost(s)); ] } else { [wq. add_all(split(work)); ] } } assert(best=best’) }
Verifying Determinism § Verify determinism of each piece. § No need to consider cross product of all interleavings. P P P
Verifying Determinism § Compositional reasoning for determinism? P P Q Q Q
Summary § “Bridge” predicates and assertions • Simple to assert natural determinism • Semantic determinism § Active Testing • Direct search based on imprecise bug reports § Verify/prove determinism • Prove exact equivalence with non deterministic sequential programs • Prove semantic equivalence between non deterministic sequential programs and sequential programs
Questions
Deterministic Assertion Library § Implemented assertion library for Java: Predicate eq = new Equals(); Deterministic. open(); Deterministic. assume(set, eq); . . . Deterministic. assert(set, eq); Deterministic. close(); § Records set to check: eq. apply(set 0, set 0’) => eq. apply(set, set’)
Checking Determinism deterministic assume Pre(s 0, s 0’) { P } assert Post(s 1, s 1’) § Run P on some number of schedules. § For every pair executions of P: and of
Outline § Motivation (3 min) § Deterministic Specification (6 min) § Experimental Evaluation (5 min) § Related Work (3 min) § Future Work + Conclusions (3 min)
- Slides: 59