HEY YOU GET OFF OF MY CLOUD EXPLORING

HEY, YOU, GET OFF OF MY CLOUD: EXPLORING INFORMATION LEAKAGE IN THIRD-PARTY COMPUTE CLOUDS Thomas Ristenpart UCSD Eran Tromer MIT Hovav Shacham UCSD Stefan Savage UCSD ACM CCS 2009 Presented by: Yun Liaw

Outline 2 Introduction Threat Model The EC 2 Service Cloud Cartography Determine Co-Residence Exploiting Placement in EC 2 Cross-VM Information Leakage Conclusions & Comments Presented by: Yun Liaw 2009/11/20

Introduction 3 Cloud Computing: The next generation infrastructure for hosting data and deploying software and services Amazon, Google, Microsoft, … Advantages: economies of scale, dynamic provisioning, low capital expenditure… In cloud infrastructure, in order to maximize the efficiency, physical resources are shared between virtual machines Here introduces the new risk! Presented by: Yun Liaw 2009/11/20

Threat Model 4 Cloud customer ↔ Malicious cloud provider The cloud provider violates customer confidentiality or integrity Insider threats within the cloud provider But this is not in the scope of this paper! Cloud Customer ↔ Non-provider-affiliated malicious parties Direct compromise A malicious party can run its instance on the same physical machines as the potential victim ← co-residence The attacker might manipulate shared physical resources to learn the victim’s confidential information Here’s this paper’s focus! Presented by: Yun Liaw 2009/11/20

Threat Model 5 Can one Determine where in the cloud infrastructure an instance is located? Can one easily determine if two instances are coresident on the same physical machine? Can an adversary launch instances that will be coresident with other user’s instance? Can an adversary exploit cross-VM information leakage once co-resident? Presented by: Yun Liaw 2009/11/20

The EC 2 Service 6 Provided by Amazon VM: Xen hypervisor[1] To monitor VMs Provide isolation between VMs Intermediating access to physical devices Domain 0 (Dom 0): A privileged VM in Xen hypervisor to manage: guest images physical resource provisioning access control right Dom 0 is used for route packets of the guest image Can be tracked by traceroute [1]P. Barham, et al. , “Xen and The Art of virtualization, ” ACM SOSP, 2003 Presented by: Yun Liaw 2009/11/20

The EC 2 Service 7 Three “degrees of freedom” in specifying the physical infrastructure of an instance Region: United State or Europe Availability Zone: Distinct locations that are engineered to be insulated from failures each region has three availability zones e. g. , us-east-1 a Instance Type: Five types: “m 1. small”, “m 1. large”, “m 1. xlarge”, “c 1. medium”, “c 1. xlarge” http: //aws. amazon. com/ec 2/#instance Presented by: Yun Liaw 2009/11/20

8 The EC 2 Service and Network Probing Each instance in EC 2 are given Internet connectivity by: External IPv 4 address and domain name Internal RFC 1918 private address and domain name e. g. , 75. 101. 210. 100 with domain name ec 2 -75 -101 -210 -100. compute-1. amazonaws. com e. g. , 10. 252. 146. 52 with domain name dom. U-12 -31 -38 -00 -8 D-C 6. compute-1. internal The DNS resolution queries from an EC 2 instance Can determine the external name of an instance Can determine the internal IP of an instance’s external IP Presented by: Yun Liaw 2009/11/20

Cloud Cartography 9 To “map” the EC 2 service to understand where the instances are located in the cloud To understand the instance creation parameter needed to attempt establishing co-residence instance Assumption: Different availability zones and instance types are likely to correspond to different internal IP ranges Validation: Create a number of EC 2 instances with different zones and types Presented by: Yun Liaw 2009/11/20

Cloud Cartography – Experiment 1 10 Experiment 1: Usingle account A to create 300 instances Each zone/type pair are with 20 instances If internal IPs are statically assigned to physical machines… Availability Presented by: Yun Liaw zones use separate physical infrastructure! 2009/11/20

Cloud Cartography – Experiment 2 11 Experiment 2: Using another account B to launch 100 instances only in zone 3 Each type are with 20 instances Result: In 100 instances in account A 92 instances had unique /24 prefix 4 of /24 prefixes had two instances, but with same type In 100 instances in account B 88 instances had unique /24 prefix 6 of /24 prefixes had two instances Only single /24 prefixes had two different type instances Presented by: Yun Liaw 2009/11/20

12 Cloud Cartography – Heuristic Rules Heuristic Rule for label /24 prefixes with zones and types: All IPs from a /16 are from the same zone A /24 inherits any included sampled instance type. If there are multiple observed instances with distinct type, we label this /24 with each types A /24 containing a Dom 0 IP only contains Dom 0 IP. We associate this /24 to the type of Dom 0’s associated instance All /24’s between two consecutive Dom 0 /24’s inherit the former’s associated type (? ) Dom 0 IPs are consistently assigned a prefix that immediately precedes the instance IPs they are associated with (? ) Presented by: Yun Liaw 2009/11/20

13 Cloud Cartography Prevention In EC 2, local IP are statically associated to zones and types randomly assign internal IP address It would make administrator’s task more challenging Cloud provider may isolate each account’s view of the internal IP address space (restrict DNS query) Presented by: Yun Liaw 2009/11/20

Determining Co-Residence 14 The two instances are likely co-resident if they are Matching Dom 0 IP address Small packet round-trip times Numerically close internal IP address Ø Network-based co-residence check p Verification: p The ground truth: If two instances can successfully transmit via a convert channel, they are co-resident p [slide Presented by: Yun Liaw 2009/11/20

15 Determining Co-Residence Experiment: 1. 2. 3. 4. 5. 6. 7. Three accounts: control, victim and probe Two instances were launched by control in each zones Victim and probe both launched 20 instances in zone 3 Performed Dom 0 IP check for each victim/probe pair If passed, let probed victim to calculate RTT Control probed probe and victim to calculate RTT Probe sent 5 -bit to victim through the covert channel Presented by: Yun Liaw 2009/11/20

16 Determining Co-Residence Experiment Result: There are 31 pairs passed the Dom 0 IP test (62 ordered pairs) 60 pairs passed convert-channel test Two failures due to a single bit error, and these two are not for the same pair of instance Result of RTT: Presented by: Yun Liaw 2009/11/20

Determining Co-Residence 17 Simple co-residence check: 1. 2. One compares the internal IP of the two instances If they are numerically close, perform Dom 0 IP test Obfuscating co-residence check Let Dom 0 not respond in traceroute Randomly assign internal IP VLAN to isolate accounts Presented by: Yun Liaw 2009/11/20

Exploiting Placement in EC 2 18 Observations from the data: Single count was never seen to have two instances that is simultaneously co-resident No more than 8 m 1. small instances were observed to be simultaneously co-resident due to the physical limitation that single physical machine can only support at most 8 m 1. small instances Sequential placement locality: two instances run sequentially are often assigned to the same machine Parallel placement locality: two instances run at roughly the same time are often assigned to the same machine Presented by: Yun Liaw 2009/11/20

19 Exploiting Placement in EC 2 – Brute-Forcing Placement Strategy: 1. 2. 3. 4. 5. Attacker enumerates a set of potential targets (should be big) Infer the zone and type of these targets Launch probe instances in the zone and type that targets belongs to Each probe instance probes each target to see if there is any co -resident the target If not, terminate the instance immediately Experiment Result: 8. 4% co-residence coverage over 1785 probe instances, probing 1686 public web servers hosted on EC 2 The first 510 launched probe instances had achieved 90% coresidence coverage … strongly sequential placement locality! Presented by: Yun Liaw 2009/11/20

20 Exploiting Placement in EC 2 – Abusing Placement Locality Strategy: Instance Flooding 1. Once the attacker observe (or trigger) the victim launch his instance… The servers in cloud often terminate when it is not in use The attacker can monitor a server’s state to observe whether the victim’s instance is re-launched Attackers can also trigger new victim instance due to the auto scaling systems 2. Attacker launching as many instances in parallel as possible, in the appropriate zone and type Presented by: Yun Liaw 2009/11/20

21 Exploiting Placement in EC 2 – Abusing Placement Locality Experiment: the effect of zone and account A victim account launch either 1, 10, 20 instance simultaneously No sooner than 5 minutes, an attacker account launch 20 instances simultaneously 2009/11/20

22 Exploiting Placement in EC 2 – Abusing Placement Locality Experiment: the effect of increased time lag 40 victim instances were launched in zone 3 throughout the experiment (36 unique machines) Every hour a set of 20 attack instances were launched and perform the co-residence check Total co-resident: # of probe instances that were co-residence with the victim at that hour New co-resident: # of victim instances that were collided with for the first time at that hour Presented by: Yun Liaw 2009/11/20

Exploiting Placement in EC 2 23 Instance flooding! But when fails? Target instance were being assigned to machines with high instance density or even full Patching placement vulnerabilities Inhibit cartography and co-residence check Let users request placement of their VMs on machines that can only be populated by VMs from their (trusted) accounts Presented by: Yun Liaw 2009/11/20

24 Cross-VM Information Leakage Time-shared caches allow an attacker to measure when other instances are experiencing computation load (through cache access) Co-residence detection Web traffic detection Keystroke timing attack Presented by: Yun Liaw 2009/11/20

25 Cross-VM Information Leakage – Prime+Trigger+Probe 1. 2. 3. 4. 5. The probing instance allocates a contiguous buffer B of b bytes Let s be the cache line size (in bytes) Prime: the probing instance read buffer B at s-byte offsets into the cache Trigger: the probing instance perform a busy-loop until the CPU’s cycle counter jumps by a large value Probe: the probing instance measure the time it takes to again read B at s-byte offsets Presented by: Yun Liaw 2009/11/20

Cross-VM Information Leakage – Load. Based Co-Residence Detection 26 We could detect co-residence when adversary has the knowledge of computational load variation on the target instance Publicly-accessible services on the target instance A priori information about load variation on the target Presented by: Yun Liaw 2009/11/20

Cross-VM Information Leakage – Load. Based Co-Residence Detection 27 Experiment: Environment: 1. 2. A web server on EC 2 “m 1. small” instance Hosting a single 1024 byte text HTML page The attacker VM took 100 load samples After 10 sec, the attacker VM took 100 samples while simultaneously making numerous HTTP request to the web server

28 Cross-VM Information Leakage – Estimating Traffic Rates HTTP traffic rate estimation to a co-resident web server experiment Perform four separate runs of 1000 cache load measurement on a web server hosted on EC 2 “m 1. small” simultaneously with Sent no HTTP request Sent 50 HTTP request per min Sent 100 HTTP request per min Sent 200 HTTP request per min Presented by: Yun Liaw 2009/11/20

29 Cross-VM Information Leakage – Keystroke Timing Attack The adversary's goal: To measure the time between keystrokes made by a victim typing a password The keystroke time can be used to perform recovery of the password[2] Assumption: on an otherwise idle machine, a spike in load corresponds to a letter being typed into the co-resident VM’s terminal [2] D. Song, et. al. , “Timing Analysis of Keystrokes and SSH Timing Attacks, ” USENIX Security Symposium, 2001 Presented by: Yun Liaw 2009/11/20

30 Cross-VM Information Leakage – Keystroke Timing Attack Experiment setup: Not on EC 2! A testbed similar to EC 2 In order to “pinned” the VM with the CPU core EC 2 occasionally migrates VM’s virtual CPUs between the machine’s cores Reason: Ensure that machine is completely idle other than the test core Allows us to know the relation between VMs involved in the attack Presented by: Yun Liaw 2009/11/20

31 Cross-VM Information Leakage – Keystroke Timing Attack Keystroke activity side channel Repeatedly perform load measurement Report a keystroke when the measurement indicates momentarily high cache usage Experiment Result: Clear signal with 5%missed keystrokes 0. 3 false triggers per second Timing resolution: 13 ms Clear difference between keys with different effects (shell command vs enter) Inhibiting side-channel attacks Blinding techniques Presented by: Yun Liaw 2009/11/20

Conclusions & Comments 32 Conclusions: Studied the risk arise from sharing physical infrastructure between mutually distrustful users Cloud cartography Co-residence detection Placement exploitation Cross-VM information leakage Comments: The risk mentioned is more likely the system-wide problems How to configure the cloud internal network to blind the infrastructure and placement? How to log the cloud behavior to support further investigation? Presented by: Yun Liaw 2009/11/20

33 Cross-VM Information Leakage – Cache-Based Covert Channel The senders idles to transmit “ 0”, and frantically access the memory to transmit “ 1” Set-associative cache: We partition the cache sets into 2 classes: “odd sets” and “even sets” The pool of cache lines is partitioned into associative sets Each memory address is mapped into a specific associative set Odd sets with odd memory addresses; even set with even addresses Differential encoding: In order to mitigate the noise in the load sample The signal is carried in the load difference between two classes The noise would be balanced between the two classes Presented by: Yun Liaw 2009/11/20

34 Cross-VM Information Leakage – Cache-Based Covert Channel Sender: (to transmit “ 0”) 1. 2. Allocates a contiguous buffer A of a bytes Reads the even addresses in A Receiver: 1. 2. 3. 4. 5. 6. Allocate a contiguous buffer B of b bytes Sleep briefly (to build up credit with Xen’s scheduler) Prime: read all B to make sure it’s fully cached Trigger: the probing instance perform a busy-loop until the CPU’s cycle counter jumps by a large value Probe: Measure the time to read all even addresses in B, likewise for the odd addresses Decide “ 0” iff the difference is positive Presented by: Yun Liaw 2009/11/20

35 Cross-VM Information Leakage – Cache-Based Covert Channel Sender: (to transmit “ 1”) 1. 2. Allocates a contiguous buffer A of a bytes Reads the odd addresses in A Receiver: 1. 2. 3. 4. 5. 6. Allocate a contiguous buffer B of b bytes Sleep briefly (to build up credit with Xen’s scheduler) Prime: read all B to make sure it’s fully cached Trigger: the probing instance perform a busy-loop until the CPU’s cycle counter jumps by a large value Probe: Measure the time to read all even addresses in B, likewise for the odd addresses Decide “ 1” iff the difference is negative Presented by: Yun Liaw 2009/11/20