Dyn Sleep Finegrained Power Management for a LatencyCritical
















- Slides: 16
Dyn. Sleep: Fine-grained Power Management for a Latency-Critical Data Center Application Chih-Hsun Chou Daniel Wong Laxmi N. Bhuyan
Outline Background & Motivation Data Center Workload Characteristics. Existing Approaches. Dyn. Sleep Prototype with Memcached Experimental Evaluation
Data Center Latency-Critical Workloads Characteristics Server utilization Lightly loaded. Short-term variability. Request processing ON/OFF execution pattern. Non-deterministic. Poor energy efficiency at low server utilization
Power Saving Opportunities Target Qo. S is defined at peak load. Low utilization servers create latency slack. Exploiting this slack for power saving. Tail latency under light load Target tail latency 100 CDF (%) 80 60 40 Latency Slack 20 0 0 100 200 300 400 500 Latency (µsec) 600 700 800
Existing Approaches DVFS: reducing the processing rate. Frequency (GHz) 2. 7 2. 4 2. 1 1. 8 1. 5 1. 2 Voltage (V) 0. 99 0. 96 0. 94 0. 92 0. 90 0. 88 Active Power (W) 3. 42 2. 93 2. 49 2. 05 1. 68 1. 31 Limited room for down scaling. 56% frequency reduction 13% voltage reduction Limited power saving. 61% power reduction Per-core control is not common. Sleep States State transition time Target Residency Power C 0 N/A (3~3. 5 W) C 1 1 μs 1. 2 W C 3 59 μs 156 μs 0. 13 W C 6 89 μs 300 μs 0 W Limited by the length of idle periods.
Observations Short idle periods cause high idle power. Traffic variability. Fine-grained control over time and space domain. Our Solution: Dyn. Sleep
Dyn. Sleep: Overview Utilizing per-core sleep state. (space domain) Postponing the request service. Transform scattered idle periods into a longer one for deep sleep state. (reduce idle power) Dynamically determine core wake-up time. Satisfy the target tail latency constraint. (time domain)
Dyn. Sleep: Example at t=A 321 t=0 WW 31 Target Tail Latencytime Target Tail Latency R 1 arrives R 2 arrives R 3 arrives
Dyn. Sleep: Power consumption behavior Power Baseline Dyn. Sleep Active Shallow sleep Deep sleep Time
Case Study: Memcached Client send requests ∙∙ Clients Libevent fd Read and Parse Data ∙ req Worker Threadresult Request Processing Worker Thread Send Response Memcached Server libevent monitors network sockets through epoll for the request arrivals. Independent threads and requests.
Memcached with Dyn. Sleep Libevent Thread l na Dyn. Sleep Calculator W sig Thread Communication up e ak l na p ee Register/Update Timer sig fd 2 fd 1 fd Libevent Sl Clients Dyn. Sleep Manager fd Read and Parse Data req Request Processing result Send Response Request Processing Thread Two separate threads. A core is woken up by wakeup signal.
Evaluation: Experiment Setup A client server and a request processing server connected over 10 G Ethernet. Intel Xeon E 5 2697 -V 2 12 -core processor. Only support per-core DFS. On-chip energy sensors with 1 KHz sampling rate.
Evaluation: Power Saving EP Power Saving 80% Dyn. Sleep DVFS DFS 40% 50% Load 60% 40% 20% 0% 10% 20% 30% 70% 80% At low to medium load, lager latency slack leads to high power saving of Dyn. Sleep. At high load, Dyn. Sleep power saving is comparative to DVFS scheme. Dyn. Sleep significantly outperforms per-core DFS scheme.
Evaluation: Latency Distribution Baseline 95 th: 187µs DFS Dyn. Sleep Target : 686µs DFS 95 th: 448µs 100% Load 0. 3 CDF 80% Dyn. Sleep 95 th: 665µs 60% 40% 20% 0% 0 200 400 Latency (µsec) 600 800 Baseline has about 500 µsec latency slack. Large gap still left in DFS(DVFS) scheme because of the limited VF down scaling. Dyn. Sleep effective close the gap by postponing request processing.
Load Tail Latency(µsec) Evaluation: Load Changes target tail latency Dyn. Sleep DFS 800 600 400 200 0 0, 8 0, 6 0, 4 0, 2 0 0 0, 5 1 1, 5 2 2, 5 3 3, 5 4 Time (s) At low load, DFS can’t fully utilize the latency slack. At high load, DFS lacks the responsiveness and frequently violate the constraint. Dyn. Sleep responds instantaneously to the load changes because of the request level updates.
Conclusion Major source of the energy inefficiency comes from the idle power. Non-deterministic and short idle periods. We propose Dyn. Sleep Reshape the idle periods pattern. Utilize deep sleep states. Dynamically wake up to meet the strict Qo. S constraint. Our memcached prototype demonstrates up to 65% power saving.