Princeton University Electrical Engineering Phase Detection and Prediction

Princeton University Electrical Engineering Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Canturk ISCI Margaret MARTONOSI SRC Student Symposium Cary, NC 2006 Oct 10, 2006

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Program Phases § Distinct and often-recurring regions of program behavior § How can we detect recurrent execution under real system variability? § How can we predict future phase patterns? § How can we leverage predicted phase behavior for workloadadaptive power management? § Can we do better than simple, reactive methods? 2 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Research Overview § Monitor application execution via specific features Application § Classify features into phases Runtime Monitoring Hardware Performance Counters Dynamic Program Flow Power Estimation Phase Analysis Dynamic Management 3 § Detect/Predict phase behavior § Apply dynamic power management guided by phase predictions § Validate with real measurements Real Measurements Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management This Talk Application § Track memory accesses per instruction (Mem/Uop) via performance counters Runtime Monitoring Hardware Performance Counters § Classify execution into phase patterns Dynamicbased on Mem/Uop rates Program Flow § Predict future behavior with the Global Phase History Table (GPHT) predictor Power Estimation§ Use phase predictions to guide dynamic Phase. Prediction Analysis voltage and frequency scaling (DVFS) Classification Phase Dynamic Management 4 Real Measurements Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management From Execution to Phases 0. 020 0. 015 0. 010 0. 005 Phases 5 4 3 2 1 0 2. 80 E+10 0. 000 2. 90 E+10 3. 00 E+10 Cycles 3. 10 E+10 3. 20 E+10 3. 30 E+10 § Assign different Mem/Uop ranges to different phases § Higher phase number more memory bound phase § Phase patterns expose available recurrence! § Simple phase definition § Resilient to system variations § Invariant to dynamic power management actions 5 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ] Mem/Uop Rate Mem/Uop

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Predicting Phases with the GPHT GPHR PPt-1 t Pt-2 … … Pt-N-1 PHT Age / Pred-n Invalid PHT Tags PHT Pt’-1 Pt’-2 … … Pt’-N Pt’+1 20 Pt’’-1 Pt’’-2 … … Pt’’-N Pt’’+1 15 : : : Pt : : : Last observed phase from performance counters : : : … … P 0 -1 P 0 P 0 … … P 0 PHT entries : GPHR depth Predicted Phase From GPHR(0) if no matching pattern From the corresponding PHT Prediction entry if matching pattern in PHT § Similar to a global history branch predictor § Implemented in OS for on-the-fly phase prediction 6 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Prediction Accuracies Prediction Accuracy (%) 100 90 80 Last. Value 70 PHT: 1024, GPHR: 8 60 PHT: 128, GPHR: 8 PHT: 64, GPHR: 8 50 PHT: 1, GPHR: 8 n gc p c_ 20 gc 0 c_ s w up cila b w is e_ re f ga gc p_ c_ re in f te gr at e gc c_ ex am pr m p_ in gc c_ 16 pa 6 rs er _r ef ap bz si ip _r 2_ ef pr og ra m m gr bz id _i ip n 2_ s bz ou ip 2_ rce gr ap h ap ic pl u_ eq in ua ke _i n cf _i m gz ip _l og 40 § Compare to reactive approaches (Last Value prediction) § GPHT performs significantly better for highly varying applications § Up to 6 X and on average 2. 4 X misprediction improvement § § 7 Good performance down to 128 PHT entries Converges to last value as PHT entries 1 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Phase Driven Dynamic Power Management § Phase definitions Memory boundedness DVFS potential § Each predicted phase Corresponding (V, f) setting § Implementation overview: 8 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Complete Example ACTUAL_PHASE PRED_PHASE (GPHT) 0. 024 0. 020 0. 016 0. 012 Phases 0. 008 5 0. 004 4 0. 000 3 § GPHT can accurately predict varying application behavior! 2 1 0 14 Power (Baseline) 12 Power [W] Mem/Uop (GPHT) § Significant power savings compared to baseline! Power (GPHT) 10 8 6 4 2 0 2. 1 BIPS (Baseline) 1. 8 BIPS (GPHT) § Negligible performance degradation! BIPS 1. 5 1. 2 0. 9 0. 6 0. 3 0 1. 5 E+09 2. 0 E+09 2. 5 E+09 3. 0 E+09 3. 5 E+09 4. 0 E+09 4. 5 E+09 5. 0 E+09 Instructions 9 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Improvement over Reactive Methods § 7% EDP improvement over reactive methods! § Comparable or less performance degradation! 10 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Conclusions § Phase characterizations help identify repetitive application behavior under real-system variability and dynamic management actions § Runtime phase predictions with the Global Phase History Table can accurately predict future application behavior § Up to 6 X and on average 2. 4 X less mispredictions than reactive approaches § Dynamic power management guided by these phase predictions help improve system power/performance efficiency § 27% EDP improvements over baseline and 7% over reactive approaches § Presented research framework and real-system experiments can guide phase-oriented characterization and dynamic adaptation applications 11 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Thanks! 12 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management EXTRAS § 1. 1) Why care about phases examples § 1. 2) Why care about pwr phases examples § 1. 3) What are different features that prev studies looked at? § 2) Experiment setup details 13 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management 1. 1) Why Care About Phases? § Characterizing execution regions E 1 14 E 2 E 3 E 4 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management 1. 1) Why Care About Phases? § Characterizing execution regions § Managing dynamic adaptation ON 15 OFF Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management 1. 1) Why Care About Phases? § Characterizing execution regions § Managing dynamic adaptation § Use current phase/behavior to predict future behavior 1 Load Refs 0. 9 0. 8 Store Misses 0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0 3 16 8 Time [s] Canturk Isci - Margaret Martonosi 13 [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management 1. 2) Why Care About Power Phases? § Useful for: § Guiding power budget / temperature limit management Uncontrolled T Enforced T Power [W] Temp. [o. C] Time [s] Slow down! § I. e. Montecito/Foxton 17 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management 1. 2) Why Care About Power Phases? § Useful for: § Guiding power budget / temperature limit management Power [W] § Power/Temperature aware scheduling Time [s] 18 Canturk Isci - Margaret Martonosi [Bellosa et al. COLP’ 03] [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management 1. 2) Why Care About Power Phases? § Useful for: § Guiding power budget / temperature limit management § Power/Temperature aware scheduling § Power balancing for multiprocessor systems/activity migration Power Task 1 Task 2 Swap hot task Core/μP 1 Core/μP 2 Speed up! Slow down! 19 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Older 20 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management This Talk § Classify application execution into phases based on HW performance counters Application Runtime Monitoring Hardware Performance Counters Dynamic Program Flow § Predict phase behavior § Apply dynamic power management guided by phase predictions § Validate with real measurements Power Estimation Phase Analysis Dynamic Management 21 Real Measurements Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]

Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management Predicting Phases with the GPHT GPHR Pt Pt-1 Pt-2 … … Pt-N PHT Age / Pred-n Invalid PHT Tags PHT Pt’-1 Pt’-2 … … Pt’-N Pt’+1 20 Pt’’-1 Pt’’-2 … … Pt’’-N Pt’’+1 15 : : : Pt : : : Last observed phase from performance counters : : : P 0 -1 P 0 P 0 … … P 0 PHT entries : GPHR depth Predicted Phase From GPHR(0) if no matching pattern From the corresponding PHT Prediction entry if matching pattern in PHT § Similar to a global history branch predictor § Implemented in OS for on-the-fly phase prediction 22 Canturk Isci - Margaret Martonosi [ SRC Student Symp’ 06 ]