Dynamic Branch Prediction with Perceptrons Daniel A Jimnez
Dynamic Branch Prediction with Perceptrons Daniel A. Jiménez, Calvin Lin The University of Texas at Austin HPCA 2001 Presented by Nils Wistoff Mentors: Firtina Can, Salami Behzad, Hasan Seminar in Computer Architecture April 29, 2021
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 2
Processor Pipeline IF 29. 04. 21 ID Issue EX Dynamic Branch Prediction with Perceptrons - HPCA 2001 WB 3
Processor Pipeline IF Instr 4 Instr 3 29. 04. 21 Instr 2 ID Issue EX WB Instr 1 Instructions enter through the processor frontend Dynamic Branch Prediction with Perceptrons - HPCA 2001 4
Processor Pipeline IF Instr 5 Instr 4 29. 04. 21 Instr 3 Instr 2 ID Issue EX WB Instr 1 Instructions enter through the processor frontend Dynamic Branch Prediction with Perceptrons - HPCA 2001 5
Processor Pipeline Instr 6 Instr 5 29. 04. 21 Instr 4 Instr 3 IF ID Instr 2 Instr 1 Issue EX Instructions enter through the processor frontend Dynamic Branch Prediction with Perceptrons - HPCA 2001 WB 6
Processor Pipeline Instr 7 Instr 6 Instr 5 Instr 4 IF ID Issue Instr 3 Instr 2 Instr 1 EX WB Instructions are executed back-to-back 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 7
Processor Pipeline Instr 8 Instr 7 Instr 6 Instr 5 IF ID Issue EX Instr 4 Instr 3 Instr 2 Instr 1 WB Instructions are executed back-to-back 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 8
Processor Pipeline Instr 9 Instr 8 Instr 7 Instr 6 IF ID Issue EX WB Instr 5 Instr 4 Instr 3 Instr 2 Instr 1 Instructions are executed back-to-back 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 9
Processor Pipeline Instr 9 Instr 8 Instr 7 IF ID Issue EX WB Instr 6 Instr 5 Instr 4 Instr 3 Instr 2 Instr 1 Goal: 1 instr/cycle 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 10
Control Hazard IF A 2 Issue EX WB A 1 Branch A or B B 2 ID Instr 1 B 1 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 11
Control Hazard IF A 3 A 2 B 2 29. 04. 21 Issue EX WB A 1 Branch A or B B 3 ID Instr 1 B 1 Dynamic Branch Prediction with Perceptrons - HPCA 2001 12
Control Hazard A 4 B 4 A 3 B 3 A 2 B 2 IF ID Branch A or B Instr 1 Issue EX WB A 1 ? B 1 IF cannot know whether to continue at A or B 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 13
Control Hazard IF A 4 B 4 A 3 B 3 A 2 B 2 ID Issue Branch A or B Instr 1 EX WB A 1 ? B 1 IF cannot know whether to continue at A or B 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 14
Control Hazard IF A 4 B 4 A 3 B 3 A 2 B 2 ID Issue EX Branch A or B Instr 1 WB A 1 ? B 1 IF cannot know whether to continue at A or B 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 15
Control Hazard IF A 4 B 4 A 3 B 3 A 2 B 2 ID Issue EX WB Branch A or B Instr 1 A 1 ? B 1 A! Branch is resolved, result is sent to IF 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 16
Control Hazard IF A 4 A 3 A 2 ID A 1 Issue EX WB Branch A or B Instr 1 Frontend can continue fetching 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 17
Control Hazard IF A 5 A 4 A 3 A 2 ID Issue EX WB Branch A or B A 1 Instr 1 3 cycles lost! 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 18
Branch Prediction A 4 B 4 A 3 B 3 29. 04. 21 A 2 B 2 IF ID Branch A or B Instr 1 Issue EX WB A 1 ? B 1 Dynamic Branch Prediction with Perceptrons - HPCA 2001 19
Branch Prediction IF ID Branch A or B Instr 1 A? A 4 A 3 A 2 A 1 B 4 B 3 B 2 B 1 Issue EX WB Speculatively fetch one branch 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 20
Branch Prediction IF ID Issue A 1 Branch A or B Instr 1 A? A 5 A 4 A 3 A 2 B 4 B 3 B 2 B 1 EX WB Speculatively fetch one branch 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 21
Branch Prediction IF ID Issue EX A 2 A 1 Branch A or B Instr 1 A? A 6 A 5 A 4 A 3 B 4 B 3 B 2 B 1 WB Speculatively fetch one branch 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 22
Branch Prediction IF ID Issue EX WB A 3 A 2 A 1 Branch A or B Instr 1 A? A 7 A 6 A 5 A 4 B 3 B 2 B 1 29. 04. 21 Branch is resolved Dynamic Branch Prediction with Perceptrons - HPCA 2001 23
Branch Prediction A 7 A 6 A 5 A 4 B 3 B 2 B 1 IF ID Issue EX WB A 3 A 2 A 1 Branch A or B Instr 1 A! Correct Prediction: We won! 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 24
Branch Prediction IF B 4 B 3 B 2 B 1 ID Issue EX WB Branch A or B Instr 1 B! Misprediction: We pay a 3 cycle penalty 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 25
The Goal of Branch Prediction A 4 B 4 A 3 B 3 A 2 B 2 IF ID Branch A or B Instr 1 Issue EX WB A 1 ? B 1 How to make a good prediction? 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 26
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 27
Executive Summary Problem Branch predictor’s accuracy is central for performance Goal Create a branch predictor that is superior to state of the art Key Idea Perceptrons are efficient in learning and predicting certain patterns over long histories Mechanism Build a branch predictor with perceptrons as central decision makers Implement a hybrid perceptron / conventional predictor Results Perceptron: 14. 7% improvement over baseline for a fixed 4 K HW budget Hybrid Predictor: Outperforms baseline consistently across benchmarks 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 28
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 29
Static Branch Prediction J. E. Smith, A Study of Branch Prediction Strategies. ISCA, May 1981 • Idea: Branches are mostly biased • e. g. Backward Taken, Forward Not Taken • Let compiler profile and arrange code • Trivial to implement in HW • Branch behaviour often not known at compile time • Only efficient for heavily biased branches 29. 04. 21 6. 25% Misprediction Rate Dynamic Branch Prediction with Perceptrons - HPCA 2001 li l 1: addi. . . bnez. . . a 0, 16 a 0, -1 a 0, l 1 30
David A. Patterson, John L. Hennessy, Computer Architecture: A Quantitative Approach, 5 th Edition. Morgan Kaufmann, 2011 Branch History Table (BHT) 0 li l 1: 1 addi. . . x bneqz. . . a 0, 16 BHT … … a 0, -1 … … PC a 0, l 1 x 01 … … MSB = 0 Predict: Not Taken Prediction: 1. Index BHT with branch address 2. Take branch if MSB of BHT entry equals 1 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 31
David A. Patterson, John L. Hennessy, Computer Architecture: A Quantitative Approach, 5 th Edition. Morgan Kaufmann, 2011 Branch History Table (BHT) 0 li l 1: 1 addi. . . x bneqz. . . a 0, 16 BHT … … a 0, -1 … … Taken! a 0, l 1 x Increment 10 … … Update (after branch is resolved): 1. Increment BHT entry if branch was taken, decrement otherwise 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 32
David A. Patterson, John L. Hennessy, Computer Architecture: A Quantitative Approach, 5 th Edition. Morgan Kaufmann, 2011 Branch History Table (BHT) 0 li l 1: 1 addi. . . x bneqz. . . a 0, 16 BHT … … a 0, -1 … … PC a 0, l 1 x 10 … … MSB = 1 Predict: Taken Prediction: 1. Index BHT with branch address 2. Take branch if MSB of BHT entry equals 1 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 33
David A. Patterson, John L. Hennessy, Computer Architecture: A Quantitative Approach, 5 th Edition. Morgan Kaufmann, 2011 Branch History Table (BHT) 0 li l 1: 1 addi. . . x bneqz. . . a 0, 16 BHT … … a 0, -1 … … Taken! a 0, l 1 x Increment 11 … … Update (after branch is resolved): 1. Increment BHT entry if branch was taken, decrement otherwise 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 34
David A. Patterson, John L. Hennessy, Computer Architecture: A Quantitative Approach, 5 th Edition. Morgan Kaufmann, 2011 Branch History Table (BHT) 0 li l 1: 1 addi. . . x bneqz. . . a 0, 16 BHT … … a 0, -1 a 0, l 1 … … PC x 11 … … MSB = 1 Predict: Taken • Very Simple Design • Not context-aware 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 35
Pattern History Table (PHT): gshare PHT Global History Register B-6 B-5 B-4 B-3 B-2 B-1 XOR PC[5] PC[4] PC[3] PC[2] Scott Mc. Farling, Combining Branch Predictors. Technical Report TN-36, Digital Western Research Laboratory, 1993 PC[1] Program Counter (Lower Bits) PC[0] … … … 00 … 10 … 01 … 10 … … Prediction Problem: PHT scales exponentially with history length 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 36
Perceptron • Takes set of features • Makes prediction based on features (Inference) • Improves over time (training) x 0 x 1 x 2 … xn y 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 37
Perceptron: Inference x 0 w 0 X x 1 w 1 X x 2 … X … w 2 Xn wn X Σ Perceptron internal state y 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 38
Perceptron: Training x 0 w 0 + wn X x 1 w 1 + wn X w 2 + x 2 … X … wn xn wn + X wn y 0 Ground Truth 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 39
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 40
Perceptron as Branch Predictor • One perceptron per branch location • Use Global History Register as input 1 • Entries are either -1 (not taken) or 1 (taken) • Constant 1 input for branch bias x 1 w 0 x 2 w 1 w 2 … xn wn • If y ≥ 0 take branch, else do not take • Once branch outcome is known, dynamically train in case of misprediction Do not take branch 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 <0 y ≥ 0 Take branch 41
Perceptron as Branch Predictor: Observations 1 • Perceptron output y contains confidence about decision • Each weight corresponds to an input and can be interpreted as follows: x 1 w 0 x 2 w 1 w 2 … xn wn • wn >> 0: Positive correlation to –nth branch outcome • wn << 0: Negative correlation to –nth branch outcome • wn ≈ 0: Little correlation to –nth branch outcome Do not take branch 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 <0 y ≥ 0 Take branch 42
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 43
Putting it Together 1. Hash branch address to select perceptron. 2. Fetch weights of selected perceptron from SRAM. 3. Compute y from weights and global history. 4. Take branch if y > 0. 5. Train perceptron after branch is resolved. 6. Write back perceptron. 29. 04. 21 Branch i Dynamic Branch Prediction with Perceptrons - HPCA 2001 History i Outcome i Prediction i Updated Weights i 44
Delay • 1 cycle to access SRAM • Approx. 2 cycles for inference • Approx. 3 cycles from branch address to prediction. • We do not want to wait that long! ~2 cycles 1 cycle 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 45
Pipelined Operation 1. When branch is encountered, use prediction from previous iteration. 2. After branch is resolved, train and write back perceptron. 3. Update global history register, concatenate outcome with branch address, select next entry. 4. Fetch weights of selected perceptron from SRAM. 5. Make prediction for next branch. 29. 04. 21 Branch i Dynamic Branch Prediction with Perceptrons - HPCA 2001 History i+1 Outcome i Predictioni+1 i Updated Weights i+1 46
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 47
Methodology • Baseline: gshare, bi-mode (PHT predictors) • Choose fixed HW budget (4 Ki. B in the following) • Generate traces of SPEC 95 and SPEC 2000 benchmarks • Simulate branch predictors’ performace for benchmark traces 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 48
Misprediction Rates Why does the perceptron perform well? Baseline Perceptron …but not always Perceptron often ouperforms PHTs… 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 49
History Length: PHT Global History Register B-6 B-5 B-4 B-3 B-2 B-1 XOR PC[5] PC[4] PC[3] PC[2] PC[1] PC[0] Program Counter (Lower Bits) 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 … … … 00 … 10 … 01 … 10 … … 50
History Length: PHT Global History Register B-7 B-6 B-5 B-4 B-3 B-2 B-1 XOR PC[6] PC[5] PC[4] PC[3] PC[2] PC[1] PC[0] Program Counter (Lower Bits) Adding one bit of history doubles PHT size 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 … … … 00 … 10 … 01 … 10 … … … … 51
History Length: Perceptron 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 52
History Length: Perceptron Adding one bit of history adds single weight 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 53
History Length: Supporting Experiment • Same HW budget for both predictors • Measure performance over different history lengths • Perceptron can score for large history lengths Perceptron’s maximum @>60 Gshare’s maximum @18 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 54
Misprediction Rates Why does the perceptron perform well? → Efficiently captures long histories When does the perceptron perform well? 29. 04. 21 Baseline Perceptron …but not always Perceptron often ouperforms PHTs… Dynamic Branch Prediction with Perceptrons - HPCA 2001 55
Linear Separability B 1: if (a) {…} B 2: if (!b) {…} B 3: if (a|b) {…} 29. 04. 21 B 1: T B 1: N B 2: T B 3: N B 2: N B 3: T B 1: if (a) {…} B 2: if (!b) {…} B 3: if (a==b) {…} Linearly separable B 1: T B 1: N B 2: T B 3: N B 3: T B 2: N B 3: T B 3: N Dynamic Branch Prediction with Perceptrons - HPCA 2001 Linearly inseparable 56
Linear Separability: Supporting Experiment • Sort benchmarks by share of linearly separable branches • Perceptron outperforms Gshare on benchmarks with many linearly separable branches 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 57
Misprediction Rates Why does the perceptron perform well? → Efficiently captures long histories When does the perceptron perform well? → When branch function is linearly separable 29. 04. 21 Baseline Perceptron …but not always Dynamic Branch Prediction with Perceptrons - HPCA 2001 Perceptron often ouperforms PHTs… 58
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 59
Hybrid Predictor • Implement both perceptron and gshare • Use a choice table to select predictor on branch Branch Address Global History Register gshare P Choice Table 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 60
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 61
Misprediction Rates Baseline Hybrid predictor consistently ouperforms PHTs 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 62
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 63
Executive Summary Problem Branch predictor’s accuracy is central for performance Goal Create a branch predictor that is superior to state of the art Key Idea Perceptrons are efficient in learning and predicting certain patterns over long histories Mechanism Build a branch predictor with perceptrons as central decision makers Implement a hybrid perceptron / conventional predictor Results Perceptron: 14. 7% improvement over baseline for a fixed 4 K HW budget Hybrid Predictor: Outperforms baseline consistently across benchmarks 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 64
Outline 1. Branch Prediction 2. Paper Summary 1. 2. 3. 4. 5. 6. 7. 8. Executive Summary Background & Related Work Key Idea: Perceptron as Branch Predictor Implementation Results Optimization: Hybrid Predictor Final Results Conclusion 3. Analysis and Discussion 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 65
Takeaways • First dynamic branch predictor using a neural network • Performs very well for class of linearly separable branches • Hybrid predictor can efficiently combine benefits of PHT and perceptron 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 66
Strengths • Simple, straight-forward idea • Effective and efficient solution • Detailed performance analysis and discussion with lots of supporting evidence 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 67
Weaknesses • Predictor performance strongly depends on benchmark. Might not always be an improvement. • High latency of perceptron • Weak area analysis • “by examining die photos, we estimate […]” • Weak timing analysis • Estimate by comparing to multiplier • “we believe that a good implementation […] will take no more than two clock cycles […]” 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 68
Source: https: //www. anandtech. com/show/10907/amd-gives-more-zen-details-ryzen-34 -ghz-nvme-neural-net-prediction-25 -mhz-boost-steps 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 69
Impact • Perceptron branch predictor known to be used in several modern processors [1, 2] • However, not many details are public • This work is one pillar of modern branch prediction research [3] [1] https: //www. anandtech. com/show/5831/amd-trinity-review-a 10 -4600 m-a-new-hope [2] https: //www. anandtech. com/show/10907/amd-gives-more-zen-details-ryzen-34 -ghz-nvme-neural-net-prediction-25 -mhz-boost-steps [3] https: //www. jilp. org/cbp 2016/ 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001 70
Questions / Comments? 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001
How could we improve the perceptron predictor so that it works for more types of branches? 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001
Do you see further usecases for neural networks in processor microarchitecture? 29. 04. 21 Dynamic Branch Prediction with Perceptrons - HPCA 2001
- Slides: 73