OS Interaction with Cache Memories Dr Sohum Sohoni

OS Interaction with Cache Memories Dr. Sohum Sohoni School of Electrical and Computer Engineering Oklahoma State University OSU ECE Computer Architecture Evaluation, Simulation and Research

Outline Technically ‘light’ talk- broad audience Background terminology and one example of current work Wild predictions about the future Many thanks to the National Science Foundation CNS #0720741 Any views presented here are my own, and not reflective of the NSF’s views and policies OSU ECE Computer Architecture Evaluation, Simulation and Research

Some Terminology CPU Caches Miss rates Locality Prefetching Context Switches ‘state’ Working set or memory footprint Process queue OSU ECE Computer Architecture Evaluation, Simulation and Research

Goal of Memory Hierarchy Low latency, high bandwidth, high capacity, low cost 300 cycles 3 GHz 2 cycles CPU L 1 Cache 2+7 = 9 cycles L 2 Cache Main Memory 16 KB Extremely fast bus (System DRAM) 512 KB Slower bus Much slower bus 512 MB OSU ECE Computer Architecture Evaluation, Simulation and Research

What Happens in a Context Switch Current process ‘state’ is saved Scheduler is invoked Next process is ‘brought in’ TLB’s are flushed L 1 cache may be flushed New process executes for its time slice Interrupt, state saved, scheduler … Effective locality gets wiped out OSU ECE Computer Architecture Evaluation, Simulation and Research

Memory pressure Effect of Multi. Programming CPU I/O CPU 20 ms CPU 30 ms 40 ms I/O CPU 50 ms 60 ms Memory pressure 10 ms I/O CPU CPU I/O 10 ms OSU ECE 20 ms CPU I/O 30 ms CPU 40 ms CPU Computer Architecture Evaluation, Simulation and Research

Problem with Multi. Programming Increasing multi-programming increases cache miss rates Loss of locality of reference Diminishing returns from multi-programming Eventual thrashing k OSU ECE Computer Architecture Evaluation, Simulation and Research

Out of Context Prefetching Reduce the negative effect of multi-programming on CPU cache performance Predict future context switches Prefetch the working set of the next process Context Switch! Prefetch Memory Pressure OSU ECE Time Computer Architecture Evaluation, Simulation and Research

OOC Prefetching Picture Main memory L 2 OOC Prefetch L 1 CPU OSU ECE Computer Architecture Evaluation, Simulation and Research 9

Context Switch Prediction CFS IDLE - FCM ORDER 7 CFS SPEC OMP - FCM ORDER 3 CFS SPEC CPU - FCM ORDER 3 CFS SPEC OMP - FCM ORDER 7 CFS SPEC CPU - FCM ORDER 7 CFS IDLE - FCM ORDER 3 O(1) SPEC CPU - FCM ORDER 3 O(1) SPEC OMP - FCM ORDER 3 CFS IDLE - LAST CFS SPEC OMP - HYSTERESIS O(1) IDLE - FCM ORDER 7 CFS IDLE - HYSTERESIS CFS SPEC CPU - HYSTERESIS O(1) SPEC CPU - LAST O(1) IDLE - LAST O(1) SPEC CPU - FCM ORDER 7 O(1) SPEC OMP - LAST O(1) IDLE - HYSTERESIS O(1) SPEC OMP - HYSTERESIS O(1) SPEC CPU - HYSTERESIS O(1) SPEC OMP - FCM ORDER 7 CFS SPEC OMP - LAST CFS SPEC CPU - LAST 35 OSU ECE 55 75 95 115 135 Prediction Rate % 155 175 Computer Architecture Evaluation, Simulation and Research 195 10

Miss Rate Improvement 60% Miss Rate Improvement 50% 40% 30% 20% 10% ef 64 r h 2 ra y po v in x 3 sp h r ta as xa la n cb m k ex so pl gc c 2 bz ip m cf 0% OSU ECE Computer Architecture Evaluation, Simulation and Research 11

THANK YOU! Q AND A OSU ECE Computer Architecture Evaluation, Simulation and Research