Dancing With Uncertainty Saa Misailovi Stelios Sidiroglou MIT

Dancing With Uncertainty Saša Misailović Stelios Sidiroglou MIT CSAIL Martin Rinard

Example Water: Simulates system of water molecules O H H O H O H O H H

Example Water: Simulates system of water molecules O H H O H O H O H H

Example Water: Simulates system of water molecules O H H O H O H O H H

Example Water: Simulates liquid water molecules O H H O H O H O H H

Example Water: Simulates system of water molecules O H H O H O H O H H

Example Water: Simulates system of water molecules O H H O H O H O H H

Dubstep Explores the effects of selectively removing synchronization

Dubstep Highlights 1. Removing locks and opportunistic barriers trade accuracy for performance 2. Automatically explores the tradeoff space induced by candidate transformations 3. Uses statistical analysis to characterize impact of transformations on accuracy

Dubstep Workflow Prepare Find Transform Analyze Navigate

Dubstep Workflow Prepare 1. Prepare representative inputs Find 2. Prepare accuracy model Transform Analyze Navigate – Output abstraction (important parts of output) – Accuracy bound (amount of tolerable error)

Dubstep Workflow Prepare Find Transform Analyze Navigate Loops with parallel constructs • Profiling: performance & memory

Dubstep Workflow Prepare Find Transform Analyze Navigate Loops with parallel constructs • Profiling: performance & memory Interf (56. 4%) Poteng (43. 4%)

Dubstep Workflow Prepare Find Transform Analyze Navigate Removing synchronization void scratch. Pad: : update. Forces (double R[3][3]) { mutex_lock(this->lock); this->H 1 force. vec. Add(R[0]); this->Oforce. vec. Add(R[1]); this->H 2 force. vec. Add(R[2]); mutex_unlock(this->lock); }

Dubstep Workflow Prepare Find Transform Analyze Navigate Removing synchronization void scratch. Pad: : update. Forces (double R[3][3]) { mutex_lock(this->lock); this->H 1 force. vec. Add(R[0]); this->Oforce. vec. Add(R[1]); this->H 2 force. vec. Add(R[2]); mutex_unlock(this->lock); }

Dubstep Workflow Prepare Find Removing synchronization void scratch. Pad: : update. Forces (double R[3][3]) { Transform Analyze Navigate this->H 1 force. vec. Add(R[0]); this->Oforce. vec. Add(R[1]); this->H 2 force. vec. Add(R[2]); }

Dubstep Workflow Prepare Opportunistic barriers Find Transform Analyze Navigate void ensemble: : interf(){ parallel_for( interf_body, 0, Num. Mol-1 ); }

Dubstep Workflow Prepare Opportunistic barriers Find Transform Analyze Navigate void ensemble: : interf(){ parallel_for( interf_body, 0, Num. Mol-1 ); }

Dubstep Workflow Prepare Opportunistic barriers Find Transform Analyze Navigate void ensemble: : interf(){ parallel_for( interf_body, 0, Num. Mol-1 ); } • Schedule threads • Execute interf_body in parallel • Wait for all threads to complete

Dubstep Workflow Prepare Opportunistic barriers Find Transform Analyze Navigate void ensemble: : interf(){ parallel_for*( interf_body, 0, Num. Mol-1 ); [Rinard, OOPSLA 2007] } • Schedule threads • Execute interf_body in parallel • Wait for half of threads to complete Instruct remaining threads to stop

Dubstep Workflow Prepare Analyze transformed program: Find • Criticality Transform • Performance Analyze • Accuracy Navigate – Memory safety, integrity – Speedup comparison – Statistical analysis

Dubstep Workflow Prepare Input Original Program Output Abstraction (Application-Specific) Find Transform < Difference Analyze Navigate c Transforme d Program δ Bound

Dubstep Workflow Prepare Find Transform Navigate the tradeoff space: • Transform and analyze one location at a time – 3 locations in water Analyze • Transform multiple locations in the same candidate program Navigate – Guided by the results of the previous step

Search Space Exploration Average Accuracy Loss vs. Speedup Relative Speedup 1. 4 LI – Synchronization Interf BI – Barrier Interf BP – Barrier Poteng 1. 3 LI+BI+BP 1. 2 LI+BP 1. 1 LI+BI BI LI 1 0 0. 02 BR 0. 04 0. 06 Baseline: original parallel program runs 6. 2 times faster than sequential on 8 cores Accuracy loss

Search Space Exploration Average Accuracy Loss vs. Speedup Relative Speedup 1. 4 LI – Synchronization Interf BI – Barrier Interf BP – Barrier Poteng 1. 3 LI+BI+BP How confident can we be LI+BP about these observations? 1. 2 1. 1 LI+BI BI LI 1 0 0. 02 BR 0. 04 0. 06 Baseline: original parallel program runs 6. 2 times faster than sequential on 8 cores Accuracy loss

Execution Reliability The probability p that the transformed program on the given input produces the result with error less than bound δ While we cannot model p, we can specify minimum acceptable reliability r

Execution Reliability Determine if program’s reliability p > r •

Execution Reliability Determine if program’s reliability p > r • How to pick N?

How Many Runs Are Enough? Procedure that determines that p >r: • Returns correct result most of the time – Wrong decision rate – Tolerance region • Quickly determines extreme (very good or bad) transformations

Statistical Analysis Sequential Probability Ratio Test

Statistical Analysis Sequential Probability Ratio Test •

Statistical Analysis Sequential Probability Ratio Test • Bound (δ) Best Transformation 0. 01 0. 05 0. 10 0. 15 LI LI LI+BI+BR

Statistical Analysis Sequential Probability Ratio Test • Bound (δ) Best Transformation 0. 01 0. 05 0. 10 0. 15 LI LI LI+BI+BR

Exploring Tradeoff Space Prepare Find Transform Start: Sequential program with for loops Transformations: • Parallel loop introduction • Synchronization, Replication Analyze Navigate Quickstep [MIT-TR-2010 -38, TECS/PEC 2012]

Exploring Tradeoff Space Prepare Find Transform Analyze Navigate Start: Program with for loops Transformations: • Skip loop iterations (multiple forms) Loop Perforation [ICSE 2010, ONWARD 2010, SAS 2011, FSE 2011]

Exploring Tradeoff Space Prepare Find Transform Start: Program with command line parameters Transformations: • Alternate function versions activated by CL parameters Analyze Navigate Dynamic Knobs [ASPLOS 2011]

Exploring Tradeoff Space Prepare Find Transform Start: Program is a tree of Map-Reduce type tasks Transformations: • Function Substitution • Reduction Sampling Analyze Navigate Nap. Red [POPL 2012]

Exploring Tradeoff Space Prepare Find Transform Start: Parallel program with for loops Transformations: • Removing Locks • Opportunistic Barriers Analyze Navigate Dubstep [Today: RACES 2012]

Reasoning About Accuracy Prepare Find Transform Analyze Navigate Exploring levels of accuracy guarantees: • Logic-based • Probabilistic • Statistical • Empirical