SIGCSE 2013 MDAT A Multithreading Debugging and Testing

SIGCSE 2013 MDAT: A Multithreading Debugging and Testing Tool Eric Larson Rochelle Palting Seattle University

Testing Multithreaded Programs is Hard! • Need to check: – Race conditions do not happen – Deadlock does not occur – Threads still run in parallel with a suitable level of concurrency • Need to run the same input with multiple OS schedules. – Little to no control over the OS schedule. – Too many schedules to exercise. • In many cases, the program appears to be working correctly. – To get a failure, need a context switch at the precise moment.

Introducing MDAT • MDAT is a multithreaded debugging and testing tool. • Designed for students learning to write concurrent programs. • Takes control over the scheduling of threads. – Only one thread is ever active at once – the OS has no control. – The scheduler forces more context switches at different points in the program. – An interactive mode allows for students to try out their own schedules. • Assists with debugging: – A trace tracks the state of threads, locks, and semaphores. – Can reproduce the schedule of a failing run.

Outline of Talk • Motivation • MDAT Overview – Scheduler – Internals – Programming Interface – Interface with Checker – Debugging Trace • Results • Conclusion and Future Work

Sample Synchronization Problem Unisex Restroom Problem: • An office contains a single restroom that can be used by both men and women but not at the same time. • Develop a solution that allows any number of men or any number of women, but not both, in the restroom at the same time. • The solution must implement the required mutual exclusion and avoid deadlock. However, the solution does not need to be fair.

Sample Incorrect Solution ENTRY SECTION if (is. Female) { pthread_mutex_lock(&female. Lock); if (male. Count != 0) sem_wait(&female. Release); female. Count++; pthread_mutex_unlock(&female. Lock); } else { pthread_mutex_lock(&male. Lock); if (female. Count != 0) sem_wait(&male. Release); male. Count++; pthread_mutex_unlock(&male. Lock); } One thread is here with female. Count = 0 and Increments female. Count = 0 Another thread is here. Reads that female. Count is 0 and skips the wait. Increments male. Count Both males and females are in the restroom at the same time!

Testing the Solution • When running the solution on Linux, nothing wrong is detected. – No deadlock. – No mutual exclusion violations (even when using a checker specific to this problem) • Why? – The OS did not generate a schedule that encountered the problem. – Exposing the error requires getting a context switch within the entry section. – The entry section is very small – well within the time slice of the OS. – Repeated runs will not help.

MDAT: Scheduler • MDAT takes control of the scheduler from the OS. – Only permits one thread to execute. – Implemented in Linux using signals and sigwait. • MDAT invokes the scheduler at the end of each simple statement. – Complex statements are broken down into one or more simple statements. • MDAT has two scheduling modes: – Random: Selects the next thread randomly. – Interactive: Allows the user to select the next thread.

MDAT: Internals 9

MDAT: Programming Interface • MDAT implements the basic synchronization primitives for locks and semaphores. • Their usage is identical to that of the Pthreads version. – Except the init functions have an additional name parameter that is used in the debugging trace. Pthreads function name MDAT function name pthread_mutex_init mdat_mutex_init pthread_mutex_lock mdat_mutex_lock pthread_mutex_unlock mdat_mutex_unlock sem_init mdat_sem_init sem_wait mdat_sem_wait sem_post mdat_sem_post 10

MDAT: Programming Interface (Additional Functions) • mdat_init: Initializes the MDAT library. • mdat_thread_start: Registers a thread. Must be called by each thread at the beginning. • mdat_thread_finish: Notifies MDAT that the thread has finished and not to schedule thread any longer. Must be called by each thread once it completes. • mdat_invoke_scheduler: Forces MDAT to invoke the scheduler and select a thread to run out of the threads that are ready (including the currently executing thread). • mdat_enter_section: Notifies MDAT that the code has entered a section. MDAT relays this information to the checker. 11

MDAT: Interface with the Checker • To use the checker, students must mark the beginning and end of the critical section using mdat_enter_section. • mdat_enter_section has three parameters: – section: name of the section – id: thread id – state: state associated with thread • At the start of the critical section, use: mdat_enter_section(“Critical”, id, is. Female); • At the exit to the critical section, use: mdat_enter_section(“Exit”, id, is. Female); 12

MDAT: Interface with the Checker • When mdat_enter_section is called, MDAT will call the checker with the same parameters. • The checker varies depending on the mutual exclusion properties of the problem. • For the unisex restroom problem: – Enter the critical section (male): Increment number of males. Check to make sure number of females is zero. – Exit the critical section (male): Decrement number of males. • MDAT will also note when a section is entered on the debugging trace. – Students can use mdat_enter_section to create their own debugging statements. 13

MDAT: Debugging Trace [Call 7] Switching from thread 6 to thread 2 Thread 2 is releasing lock male. Mutex Transferring lock male. Mutex from thread 2 to thread 4 **************************************** |THREAD ID |LOCATION |STATUS |WAITING ON | |0 |7 |waiting-lock |male. Mutex | |1 |3 |waiting-lock |female. Mutex | |2 |9 |running | | |3 |21 |completed | | |4 |17 |ready | | |5 |6 |waiting-sem |gender. Block | |6 |7 |ready | | |7 |21 |completed | | |8 |7 |waiting-lock |male. Mutex | |9 |21 |completed | | |10 |17 |waiting-lock |male. Mutex | ---------------------------------------|LOCK NAME |STATUS |WAITING THREADS | |female. Mutex |held by 5 |1 | |male. Mutex |held by 4 |0 8 10 | ---------------------------------------|SEMAPHORE NAME |VALUE |WAITING THREADS | |occupancy |2 | | |gender. Block |-1 |5 | **************************************** 14

Results • Students in two offerings of the course were assigned to implement two variants of the unisex restroom. – Original: No males and females at the same time. – Restricted: Same as original but at most 4 people can be in the restroom at once. • Fall 2011 offering (21 submissions): – Did not use MDAT. – Did have access to a mutual exclusion checker. • Fall 2012 offering (28 submissions): – Did use MDAT. – The mutual exclusion checker was separate but automated as part of the Run MDAT script (as described in paper). • Each of the 49 submissions was run with 200 randomly generated schedules with 12 threads and 10 rounds. 15

Results: Quality of Submissions Fall 2011 Fall 2012 (w/o MDAT) (MDAT) No errors (correct solution) 6 23 No errors (incorrect solution) 0 1 Deadlock (all 200 runs) 10 0 Deadlock (some runs) 1 0 Mutual exclusion violation (all 200 runs) 1 1 Mutual exclusion violation (some runs) 0 2 Both deadlock & mutual exclusion 2 1 Hung due to incorrect spin loop 1 0 16

Results: Using MDAT YES NO Did MDAT detect deadlock? 24 2 Did MDAT detect a case where men and women used the restroom at the same time? 16 9 Did MDAT detect a case where more than four people used the restroom at once? 8 16 Did you look at the output trace? 22 4 Did you use interactive mode? 17 9 Usefulness of … (Very Useless) (Very Useful) 1 2 3 4 5 output trace 1 3 7 4 7 interactive mode 0 2 0 7 7 17

Summary • MDAT helps students test multithreaded programs. – Students found errors in their programs. – The number of incorrect solution drastically reduced. • MDAT helps student debug multithreaded programs. – Students were able to fix the errors found. – Students found the interactive mode helpful. 18

Future Work • Improve the trace facility so it is easier to understand. – Integrate the checker (ongoing) – Reduce the density of the trace (needs work) • Improve the scheduler. – Learn the limits of random scheduling – Focus on test generation. • Allow for MDAT to use the actual locking and semaphore functions. – Could use MDAT on existing programs. 19

Questions?