Micro Scope Enabling Microarchitectural Replay Attacks Dimitrios Skarlatos

Micro. Scope: Enabling Microarchitectural Replay Attacks Dimitrios Skarlatos, Mengjia Yan, Bhargava Gopireddy, Read Sprabery, Josep Torrellas, and Christopher W. Fletcher University of Illinois at Urbana-Champaign

Overview

The Era of Side-Channels Processor Core Port 0 Spectre’ 18 ROB LSQ 4 K-Alliasing’ 18 Scheduler Port 2 Port 3 Port 1 Port. Smash’ 18 TLBLeed’ 18 Branch ALU TLB Branch Predictors’ 16 Non-Inclusive LLC’ 19 L 2 L 3 L 2 SGX FPUFPU’ 15 Prime+Probe[13] Subnormal D-L 1 TLB I-L 1 Contention’ 13 Main Memory DRAMA’ 16

How much can leak over side channels? Victim: Attacker: if (secret) use resource else don’t use resource for. . t 1 = time() use resource t 2 = time() • Need repeated measurements to be confident Denoise • However, many applications run only once Attacker gets 1 measurement • Can attackers really extract secrets?

Contribution: Microarchitectural Replay Attacks • Attacker leverages speculative execution • To repeatedly replay a snippet of victim code • That runs only once } Primitive to denoise arbitrary side channels Victim: ld addr // “replay handle” … ld secret // secret the attacker tries to leak Memory operation that will cause a squash and re-execute

Contribution: Microarchitectural Replay Attacks Time ld addr: Issue Replay Handle Long Latency Event

Contribution: Microarchitectural Replay Attacks Time ld addr: ld secret: Issue Replay Handle Speculative Execution of Secret Long Latency Event

Contribution: Microarchitectural Replay Attacks Time ld addr: ld secret: Issue Replay Handle Long Latency Event Speculative Execution of Secret Squash Event Squash Clear State

Contribution: Microarchitectural Replay Attacks ld addr: ld secret: Issue Replay Handle Long Latency Event Speculative Execution of Secret Cause Shared Resource Contention & Monitor Squash Event Squash Clear State Replay!!

Background

Page Tables Background Virtual Address A CR 3 + 47 … 39 38 … 30 29 … 21 20 … 12 11 … 0 9 -bits Page Offset pgd_t PGD + pud_t PUD + pmd_t + PMD • Page tables stored in memory • On a TLB Miss “page walk” = memory accesses pte_t PTE • Each step of page walk = cache hit/miss. • Page walk cache (PWC): hardware cache of translations • If Present bit in pte_t is cleared Page Fault, invoke OS TLB Entry

Trusted Computing with SGX • • Designed to run sensitive applications in the cloud Do not trust OS/Hypervisor cannot introspect/tamper enclave Unfortunately, OS/Hypervisor still manages demand paging Attack Surface With Enclaves App App Operating System Hypervisor Hardware Attack Surface

Threat Model

Threat Model In SGX the OS is responsible • Demand paging Attacker (OS) can: • Evict TLB entries • Evict page walk cache entries • Monitor side channels

Micro. Scope: A framework to exploit microarchitectural replay attacks

Attack Examples Victim Code 1. 2. 3. 4. 5. //public address handle(pub_addr); . . . transmit(secret); . . . Loop Victim Code: 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. mem. Op(pub_addr. B); 7. .

Terminology Victim Code 1. 2. 3. 4. 5. //public address handle(pub_addr); . . . transmit(secret); . . . Replay handle: • Load to a public address (known to OS) Transmitter: • Any instruction(s) whose execution reveals secret through some side channel • Occurs < ROB length from Replay Handle

Timeline of a Micro. Scope Attack - Setup Attacker Time Victim

Timeline of a Micro. Scope Attack - Setup Clear PTE Present Bit of Replay Handle Attack Setup Attacker Time Victim

Timeline of a Micro. Scope Attack - Setup Clear PTE Present Bit of Replay Handle Attack Setup Attacker Flush Replay Handle Page Table Entries Time Victim

Timeline of a Micro. Scope Attack - Setup Clear PTE Present Bit of Replay Handle Attack Setup Attacker Flush Replay Handle Page Table Entries Flush Replay Handle TLB Entry Time Victim

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Attacker Time Issue Replay Handle Victim

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Attacker Time Issue Replay L 1 TLB Handle Miss Victim

Timeline of a Micro. Scope Attack Setup handle(pub_addr): transmit(secret): Attacker Time Issue Replay L 1 TLB L 2 TLB Handle Miss Speculative Execution of Transmitter Victim

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Time Issue Replay L 1 TLB L 2 TLB PWC Handle Miss Speculative Execution of Transmitter transmit(secret): Attacker Victim

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Time Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim

Timeline of a Micro. Scope Attack Tune speculative execution duration with: Cache Hit or Miss Attack Setup handle(pub_addr): Time Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Time Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Page Fault

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Time Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Page Fault Squash

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Time Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Page Fault Squash OS Invocation

Timeline of a Micro. Scope Attack Page Fault Handler Attack Setup handle(pub_addr): Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Page Fault Squash OS Invocation

Timeline of a Micro. Scope Attack Page Fault Handler Attack Setup handle(pub_addr): Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Flush Replay Handle Page Table Entries Page Fault Squash OS Invocation

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Page Fault Squash OS Invocation Replay!!

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Page Fault Squash OS Invocation

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Handle Miss Walk transmit(secret): Attacker Speculative Execution of Transmitter Victim Page Fault Squash OS Invocation

Timeline of a Micro. Scope Attack Setup handle(pub_addr): Issue Replay L 1 TLB L 2 TLB PWC PGD PUD PMD PTE Miss Walk Handle Miss transmit(secret): Speculative Execution of Transmitter Cause Shared Resource Contention & Monitor Attacker Victim Attacker Monitor/Contention thread Page Fault Squash OS Invocation Replay!!

Loop Example

Loop Example Victim: 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. mem. Op(pub_addr. B); 7. . • Goal: • Leak secret in every iteration • Challenge: • Monitor window of a replay handle < ROB length

Loop Example Victim: 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. pivot(pub_addr. B); 7. . Replay Handle Transmit Code Instruction used to pivot around transmit instructions Pivot Instruction

In a Loop Dynamic code Static Code 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. pivot(pub_addr. B); 7. . Replay Handle Transmit Code Iteration i=0 Iteration i=1 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Pivot Instruction handle(pub_addr. A); . . . transmit(secret[0]); . . . pivot(pub_addr. B); . . . handle(pub_addr. A); . . . transmit(secret[1]); . . . pivot(pub_addr. B); Monitor window

In a Loop Dynamic code Static Code 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. pivot(pub_addr. B); 7. . Replay Handle Transmit Code 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Pivot Instruction handle(pub_addr. A); . . . transmit(secret[0]); . . . pivot(pub_addr. B); . . . handle(pub_addr. A); . . . transmit(secret[1]); . . . pivot(pub_addr. B); Pivot page fault

In a Loop Dynamic code Static Code 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. pivot(pub_addr. B); 7. . Replay Handle Transmit Code 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Pivot Instruction handle(pub_addr. A); . . . transmit(secret[0]); . . . pivot(pub_addr. B); . . . handle(pub_addr. A); . . . transmit(secret[1]); . . . pivot(pub_addr. B); Retired instructions Cause page fault on pivot

In a Loop Dynamic code Static Code 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. pivot(pub_addr. B); 7. . Replay Handle Transmit Code 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Pivot Instruction handle(pub_addr. A); . . . transmit(secret[0]); . . . pivot(pub_addr. B); . . . handle(pub_addr. A); . . . transmit(secret[1]); . . . pivot(pub_addr. B); Retired instructions Cause page fault in new replay handle

In a Loop Dynamic code Static Code 1. for i in. . . 2. handle(pub_addr. A); 3. . 4. transmit(secret[i]); 5. . 6. pivot(pub_addr. B); 7. . 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. handle(pub_addr. A); . . . transmit(secret[0]); . . . pivot(pub_addr. B); . . . handle(pub_addr. A); . . . transmit(secret[1]); . . . pivot(pub_addr. B); Retired Instructions New Monitor Window Use this idea to denoise side channels on AES (see paper) Replay Handle Transmit Code Pivot Instruction

Port Contention Attack

Port Contention Attack with Micro. Scope Victim: Attacker: 1. Setup replay attack 1. handle(pub_addr. A); 2. … 2. if (secret) 3. for … 3. victim_mul() t 1 = time() 4. else 4. 5. attacker_div() 5. victim_div() 6. t 2 =time() 7. } Replay Handle Transmit Code No Loop! Attacker Code

Port Contention Attack with Micro. Scope Victim: Victim Thread 1. handle(pub_addr. A); 2. if (secret) 3. victim_mul() 4. else 5. victim_div() Replay Handle Decode INT ALU Transmit Code Attacker Code FP DIV Attacker Thread Attacker: 1. 2. 3. 4. 5. 6. 7. Setup replay attack … for … t 1 = time() attacker_div() t 2 =time() }

Port Contention Attack with Micro. Scope Victim: Victim Thread 1. handle(pub_addr. A); 2. if (secret) 3. victim_mul() 4. else 5. victim_div() Replay Handle Decode INT ALU Transmit Code Attacker Code FP DIV Attacker Thread Attacker: 1. 2. 3. 4. 5. 6. 7. Setup replay attack … for … t 1 = time() attacker_div() t 2 =time() }

Port Contention Attack with Micro. Scope Victim: Victim Thread 1. handle(pub_addr. A); 2. if (secret) 3. victim_mul() 4. else 5. victim_div() Decode INT ALU FP DIV 10, 000 1 Replay Handle Transmit Code Attacker Thread Attacker: 1. 2. 3. 4. 5. 6. 7. Setup replay attack … for … t 1 = time() attacker_div() t 2 =time() }

Port Contention Results • Victim and attacker run on adjacent SMT contexts of same core • Attacker performs divisions in a loop and measures time • Victim performs two multiplications without a loop • Victim performs two divisions without a loop

Generalizing Microarchitectural Replay Attacks

Microarchitectural Replay Attacks Attacker Trigger Replay? Strategy Secret 3 2 3 4 Replay Handle 2 Measure 1 1 Side Channels? ed y pla de e R Co Window 4 Victim This work: Replay Handle Page fault-inducing load Replayed Code Leaky instruction Changing each can result in Side Channel uarch structures different attacks!! Attacker strategy Page fault until denoise

More On the Paper • Other attack examples • Micro. Scope framework implementation • Attacks on AES • Countermeasures discussion

Open source! • Micro. Scope Framework • Denoise nearly arbitrary microarchitectural side channels using page faults • Only a single run of the victim • Demonstrate attacks on notoriously noisy side channels • Detect the presence or absence of two divide instructions • Single-step and denoise cache-based attacks on AES https: //github. com/dskarlatos/Micro. Scope

Takeaways Microarchitectural Replay Attacks New class of attacks • Opens large new attack surface (noisy side channels) • Implications for integrity, TSX, physical side channels • Exploits core microarchitectural feature • Dynamic instructions can be replayed through controlled squashes! • We need new security properties to mitigate these attacks!