Testing Concurrent Programs A 7 Minute JargonFree Introduction

Testing Concurrent Programs, A 7 -Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010

Computer Annoyances • Program bugs • Slow computers 2

Unit Testing • Test program parts, not the whole program – Test parts individually, then together – Smaller parts larger parts • Unit tests are programs – Compare expected result to actual result: assert. Equals(12, multiply(3, 4)); 3

Moore’s Law Adopted from Sutter 2009 4

Existing Approaches Fail • Simple Unit Testing Frameworks – Example: JUnit – Assume program is single-threaded – Many errors go undetected Main thread: Other thread: detected not detected 5

Problem: Non-Determinism • Synchronization points – Information exchanged between threads – Order of threads reaching sync. points is non-deterministic 6

Problem: Non-Determinism • Synchronization points – Information exchanged between threads – Order of threads reaching sync. points is non-deterministic – Program could succeed in one order but fail in another 7

Comprehensive Tools: Costly • Model checking or schedule-based execution – Examples: JPF, Exit. Block – Test all arrangements – Too costly t s N = # of threads = # of sync. points = # of arrangements Longer than the universe exists! (if N=1 1 minute) 8

Probabilistic Approach • Insert random delays before synchronization points to modify the order • Run tests many times with different delays 9

Probabilistic Approach • Insert random delays before synchronization points to modify the order • Run tests many times with different delays delay 10

(Proposed) Contributions • Probabilistic Approach – Random delays at synchronization points • Call Graph Analysis – Determine which parts of the program may interact – Only insert delays where necessary • Improved JUnit Framework – Exception handler for all threads – Ensures that all errors in all threads are detected 11

Evaluation • Tested Dr. Java with Improved JUnit – Number of unit tests: – Previously unknown problems: – Slowdown: ~1 percent 900 20 • Other parts still under development 12

Conclusion • Unit testing of concurrent programs will become more important • Concutest helps… – by improving JUnit’s abilities – by using probabilistic techniques – by using call graph analysis to reduce the number of synchronization points 13

Image Attribution 1. 2. 3. 4. Left image on Computer Annoyances: Adapted from Microsoft Graph on Moore’s Law: Adapted from Herb Sutter 2009 Top image on Multi-Cores and Concurrency: Adapted from Brian Goetz et al. 2006, Addison Wesley Bottom image on Multi-Cores and Concurrency: Caption Fridays 14

Extra Slides 15

Multi-Cores and Concurrency • Recent trends – Clock speed stagnated – Transistors still increasing – Reason: multi-core CPUs • Implication – Must use concurrency to benefit from modern CPUs – Concurrent programming is difficult 16

Number of Arrangements • Number of schedules (N) – t: # of threads, s: # of slices per thread – Note: s is # of slices (between synchronization points) 17

Number of Arrangements Product of s-combinations For thread 1: choose s out of ts time slices For thread 2: choose s out of ts-s time slices … For thread t-1: choose s out of 2 s time slices For thread t-1: choose s out of s time slices Writing s-combinations using factorial Cancel out terms in denominator and next numerator Left with (ts)! in numerator and t numerators with s! 18