ECICache A HighEndurance and CostEfficient IO Caching Scheme

ECI-Cache: A High-Endurance and Cost-Efficient I/O Caching Scheme for Virtualized Platforms Saba Ahmadian 1, 2, Onur Mutlu 3, 4, Hossein Asadi 1, 2 1 2 3 4 Presenter: Saugata Ghose (Carnegie Mellon University)

Introduction: Performance Gap 1 Between CPUs and I/O Devices Data centers run an increasing number of data-intensive applications Facebook in 2014: 600 TB data per day Magnetic hard drive (HDD) latency not improving Storage subsystem performance bottleneck IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory express; IBM, FAST 2010

Trade-offs in Modern Storage Devices 2 EMC Corp. , 2013 Magnetic HDDs: Low Performance, Low Cost, Larger Capacity Solid-State Drives (SSDs): High Performance, High Cost, Smaller Capacity Writes Slower than Reads, Limited Endurance

3 Hybrid Storage Systems: Combine HDDs and SSDs for Best of Both Worlds Data is Highly Skewed. Cache Hot (Active) Data on At High-Performance SSD of Data Any Given Time, Only a Small Percentage is Active Store All Other Data in Low Cost HDD HOT SSD COLD HDD How much of the SSD cache should we allocate to each application in a data center machine?

Calculating SSD Cache Size 4 Today: use reuse distance to determine how much cache to allocate to each VM Time between two accesses to the same block Achieves a balance between allocated cache size and cache hit rate Used by state-of-the-art I/O cache managers Challenge: SSD write behavior Much slower than reads More writes decrease the SSD lifetime

Traditional Reuse Distance (TRD) # of reqs RD 0: 2 RD 2: 2 RD 3: 1 RD 7: 1 Access n Address A 3 Reuse Distance (RD) n W W 7 W R R W 2 A 0 0 0 Time R 2 A 1 R 3 A 2 W 5 W

The Problem with Reuse Distance Write 6 requests do not fall on the critical path In SSDs, read requests (on the critical path) can be delayed by long-latency write requests TRD does not consider the request type Write After Write (WAW) first write not used, but incurs latency/endurance cost Write After Read (WAR) unclear if newly-written data will be reused later Read After Read (RAR)/Read After Write (RAW) data is needed by the W application useful W 2 New Metric: Useful Reuse Distance Consider only requests that we know will be reused

7 Useful Reuse Distance (URD) # of reqs 1 RD 0: 2 1 1 RD 2: 2 1 RD 3: 1 0 RD 7: 1 0 Access n Address A 3 Reuse Distance (RD) n W W 7 W 3 A 2 R R W W 2 A 0 0 0 Time R 2 A 1 R W

8 ECI-Cache High-Endurance, Cost-Efficient I/O Cache Hypervisor-based SSD cache manager Implemented as a working QEMU module Allocates per-VM cache space dynamically using URD Assigns per-VM write policies adaptively

ECI-Cache Management For 9 each VM Collect storage traces from prior interval Calculate fraction of reuse for each type: RAR, RAW, WAR, WAW Determine If write policy WAW + WAR > threshold, disable cache writes Otherwise, Calculate cache uses writebacks Useful Reuse Distance Adjust cache sizes (see the paper) Assign policies

Evaluation: Experimental Setup All 10 experiments run on a real platform HP Proliant DL 380 Gen. 5, 8 Xeon CPUs, 16 GB RAM 4 x HDD: 146 GB SAS 10 K HP RAID 5 (3+1) SSD: 128 GB SATA Samsung 850 Pro QEMU 16 Hypervisor on Centos 7 (Kernel: 3. 10. 0 -327) running VMs (OS: Ubuntu 15. 04 and Centos 7) Applications: Compare MSR traces from SNIA ECI-Cache to Centaur [Koller+ ICAC ’ 15], best prior I/O cache manager

11 ECI-Cache-Perf-per-Cost rs rc h_ 2 m ds _1 us r_ 0 1 sr c 2 _0 w eb _0 pr n_ sr c_ 1 0 rs rc h_ 0 pr xy _ 0 pr oj _ 0 ds _ m _1 hm _0 ts st g_ 1 eb w de w _1 1 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0 v_ 0 Perf-per-Cost (Normalized) Experiments: Performance-Per-Cost Improvement Centaur-Perf-per-Cost è ECI-Cache improves perf-per-cost by more than 30% ü Much smaller cache space allocation by ECI-Cache ü Higher hit rate for ECI-Cache

12 ECI-Cache-Perf rs rc h_ 2 m ds _1 us r_ 0 1 sr c 2 _0 w eb _0 pr n_ sr c_ 1 0 rs rc h_ 0 pr xy _ 0 pr oj _ 0 ds _ m _1 hm _0 ts st g_ 1 _1 eb w de v_ 0 1 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0 w Performance (Normalized) Experiments: Performance Improvement Centaur-Perf è ECI-Cache improves performance by 17%

13 Centaur rs rc h_ 2 m ds _1 us r_ 0 eb _0 0 w sr c 2 _ 1 pr n_ sr c_ 1 0 rs rc h_ 0 pr xy _ 0 pr oj _ 0 ds _ m _1 hm _0 ts st g_ 1 _1 eb w de v_ 0 1 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0 w Writes on Cache (Normalized) Experiments: Endurance (Lifetime) Improvement ECI-Cache è ECI-Cache improves SSD lifetime by 65%

Conclusion 14 I/O is a performance bottleneck in virtualized platforms Improve I/O performance using an SSD cache SSD writes introduce long latencies, lower endurance URD: Useful Reuse Distance New metric: count reuse only when next access is read ECI-Cache: Hypervisor-based I/O cache - Per-VM URD-based cache sizing - Per-VM write policy Results on real system Improved Performance (17%), Performance-per-Cost (30%), and Endurance (65%)

ECI-Cache: A High-Endurance and Cost-Efficient I/O Caching Scheme for Virtualized Platforms Saba Ahmadian 1, 2, Onur Mutlu 3, 4, Hossein Asadi 1, 2 1 2 3 4 Presenter: Saugata Ghose (Carnegie Mellon University)

16 Backup Slides

Challenge for Hybrid Storage: Closing the Performance Gap Data is Highly Skewed At Time, Only Hot. Any Data. Given. Cold Data Active Data on High Performance SSD a Small Percentage of Data is Active Inactive Data on Low Cost HDD ACTIVE SSD INACTIV E HDD Small Size Hot Data SSD Large Size Cold Data HDD EMC Corp. , 2015. IMEX Research Cloud Infrastructure Report 2009 -2011. 17

Previous Cache Size Estimation Approaches Reuse Intensity 18 Mostly capable for CPU caches WSS (Working Set Size) Doesn’t measure data reuse Indicates required space for acceptable performance Compared to Previous Approaches, Reuse Distance Based Provides Most Effective Cache Size Reuse Distance Based Estimation Existing Reuse Distance Calculation Trade-off between allocated size and miss ratio Approaches Not Miss Ratio Curves (MRCs) Request Type Consider Miss ratio as a function of cache size Predict future Perf. Our Goal: Propose a Novel Reuse Distance Locality Calculation Scheme that Considers: 1) Distance and Spatial 2) Type Requests Temporal (in selected timeof intervals)

Introduction: Why Virtualization? High 19 Resource Utilization Delivering Providing Each high performance for each VM System Isolation VM limited to its own resources Providing Abstraction Dynamically adjustable Storage Hypervisor Network Computing Resources XEN VMWare ESX QEMU Virtual. Box Available in: http: //www. arumtec. net/fr/outils-virtualisation/outils-de-virtualisation/vmware-esxi-5/presentation-de-vmware-esxi-5, Accessed: Oct, 2015.

Motivation: HDDs vs. SSDs WD Black SATA HDD (4 TB) Slow (180 MB/s, IOPS: 140) Power Hungry (Active: 9 w, Idle: 8 w) Heavy (780 g) Noisy MTBF < 1 Mh Inexpensive (0. 06 $/G) Unlimited Writes 20 Samsung 850 PRO SATA SSD (2 TB) Fast (550 MB/s, IOPS > 50 K) Low Power (Active: 3 w, Idle: 60 mw) Light (66 g) Very Low Noise MTBF: 2 Mh Expensive (0. 1 $/G) Limited Writes http: //www. storagereview. com/, Accessed: Oct, 2015

Challenge for Hybrid Storages: IO Caching ü Balance Workload ü Handle IO Bursts 21 Active Data on High Performance SSD ACTIVE SSD INACTIV E HDD Inactive Data on Low Cost HDD Hot Data Cold Data

22 How URD based cache works? (cont. ) Example#3 (URD): 1 2 3 5 6 Write 1 Write 2 Read 1 Read 3 Read 5 Read 6 Read 2 Access Hit Max. URD = 4 Cache Size = 5 blk 1 RD(WA*) < RD(RA*) TRD �� RD(RA*), URD �� RD(RA*) TRD = URD 2 RD(WA*) > RD(RA*) TRD �� RD(WA*), URD �� RD(RA*) TRD > URD

Architecture of ECI-Cache Resides As in Path of VMs IO Requests a Module of Hypervisor Kernel Full Control on VMs IO Requests ü Monitor ü Extracts IO traces ü Device block layer ü blktrace ü Analyzer ü Estimates size (URD) ü Assigns write policy ü Actuator ü Performs configs. 23

Experiments: Cache Allocation (Unlimited SSD Space) Cache Allocation for VMs by Centaur (TRD) 24 Cache Allocation for VMs by ECI-Cache (URD) ü Much Smaller Cache Space Allocation by ECI-Cache ü Similar Hit Ratio by ECI-Cache ü ECI-Cache Improves Perf-Per-Cost by 30%

Experiments: Cache Allocation (Limited SSD Space Infeasible) Cache Allocation for VMs by Centaur (TRD) 25 Cache Allocation for VMs by ECI-Cache (URD) ü Much Smaller Cache Space Allocation by ECI-Cache ü Higher Hit Ratio by ECI-Cache è ECI-Cache Improves Perf-Per-Cost by More than 30% è ECI-Cache Improves Performance by 17%

Alternatives For SSD Caching 26 (cont. ) VM-Based SSD Caching VM has best knowledge and full control of Cache Isolation VM 1 VM 2 VMn SSD SSD Guest OS should modify and create cache size not dynamically adjustable Hypervisor Cache HDD HDD

Alternatives For SSD Caching 27 (cont. ) Storage System-Level SSD Caching VM 1 Easy to use Not transparent for VMs Performance management cannot be guaranteed VM 2 VMn Hypervisor SSD HDD HDD

Alternatives For SSD Caching 28 (cont. ) Hypervisor-Based SSD Caching Transparent for VMs Hypervisor full control Hypervisor is aware of VMs workload VM 1 VM 2 VMn Hypervisor SSD HDD HDD

Previous Work: S-CAVE Hypervisor-Based Granularity: Policy: SSD Cache Fixed-Sized Write Through Dynamic Size Management Estimate Cache Demand For Each VM: 29 Working Set Size (WSS) Luo, Tian, et al. "S-cave: Effective ssd caching to improve virtual machine storage performance. " Proceedings of the 22 nd international conference on Parallel Architectures and Compilation Techniques (PACT), 2013.

Previous Work: v. Cache. Share Hypervisor-Based Granularity: Policy: SSD Cache Fixed-Sized Write Around (RO) Dynamic Cache Size Management Estimate Cache Demand for each VM: Cache Utility CU = l × RR × (H(c) + RI) Meng, Fei, et al. "v. Cache. Share: automated server flash cache space management in a virtualization environment. “ USENIX ATC. 2014. 45

Previous Work: Centaur Hypervisor-Based Granularity: Policy: SSD Cache Fixed-Sized Write Back Dynamic Cache Size Management Estimate Cache Demand for each VM: Reuse MRCs Distance (RD) 46

Previous Work: Cloud. Cache Hypervisor-Based Policy: VMs 47 SSD Cache Write Back Migration Performance Endurance Estimate Reuse Cache Demand for each VM: Working Set Size (RWSS) Temporal Locality Arteaga, Dulcardo, et al. "Cloud. Cache: On-demand Flash Cache Management for Cloud Computing. " FAST. 2016.

Latency Improvement 48

Operations on VMs Workload 49

Operations on VMs Workload 50

51 Previous Work SSD Caching (in Virtualized Platforms) com rc me com co Centaur Cloud Cache VMWare, ESX XEN ESX ESX Global Global Dynamic. Partitioned Host-side Host-side Host-side WSS Reuse Intensity Reuse Distance Reuse WSS Dynamic - Partitioned Static/Global NOMS’ 14 KVM/QEMU EMC’ 12 VMWare, Linux, Windows FAST’ 16 v. Cache Share ICAC’ 15 S-CAVE ATC’ 14 Effective SSD Caching PACT’ 13 v. FRM TECS’ 14 Fusion io. Turbine Fusion-io’ 11 com Net. App Mercury l rcia e mm MSST’ 12 EMC VFCache l rcia e m ial l rcia me

Previous Work: 52 Dynamic Cache Space Allocation S-CAVE [PACT’ 13] Hypervisor-based Policy: Size SSD cache Write Through estimation: Working Set Size (WSS) v. Cache. Share [ATC’ 14] Hypervisor-based Policy: Size SSD cache Write Around (RO) estimation: Reuse Intensity CU = l × RR × (H(c) + RI) [1] Luo, Tian, et al. "S-cave: Effective ssd caching to improve virtual machine storage performance. " Proceedings of the 22 nd international conference on Parallel Architectures and Compilation Techniques (PACT), 2013. [2] Meng, Fei, et al. "v. Cache. Share: automated server flash cache space management in a virtualization environment. “ USENIX ATC. 2014.

Previous Work (Cont. ): 53 Dynamic Cache Space Allocation Centaur [ICAC’ 15] Hypervisor-based SSD cache Policy: Write Back Size estimation: Reuse Distance (RD) MRCs Cloud. Cache [FAST’ 16] Hypervisor-based SSD cache Policy: Write Back VMs migration Perf. + Endurance Size estimation: Reuse Working Set Size (RWSS) Temporal locality Arteaga, Dulcardo, et al. "Cloud. Cache: On-demand Flash Cache Management for Cloud Computing. " FAST. 2016.