Dyn Sleep Finegrained Power Management for a LatencyCritical

  • Slides: 16
Download presentation
Dyn. Sleep: Fine-grained Power Management for a Latency-Critical Data Center Application Chih-Hsun Chou Daniel

Dyn. Sleep: Fine-grained Power Management for a Latency-Critical Data Center Application Chih-Hsun Chou Daniel Wong Laxmi N. Bhuyan

Outline Background & Motivation Data Center Workload Characteristics. Existing Approaches. Dyn. Sleep Prototype with

Outline Background & Motivation Data Center Workload Characteristics. Existing Approaches. Dyn. Sleep Prototype with Memcached Experimental Evaluation

Data Center Latency-Critical Workloads Characteristics Server utilization Lightly loaded. Short-term variability. Request processing ON/OFF

Data Center Latency-Critical Workloads Characteristics Server utilization Lightly loaded. Short-term variability. Request processing ON/OFF execution pattern. Non-deterministic. Poor energy efficiency at low server utilization

Power Saving Opportunities Target Qo. S is defined at peak load. Low utilization servers

Power Saving Opportunities Target Qo. S is defined at peak load. Low utilization servers create latency slack. Exploiting this slack for power saving. Tail latency under light load Target tail latency 100 CDF (%) 80 60 40 Latency Slack 20 0 0 100 200 300 400 500 Latency (µsec) 600 700 800

Existing Approaches DVFS: reducing the processing rate. Frequency (GHz) 2. 7 2. 4 2.

Existing Approaches DVFS: reducing the processing rate. Frequency (GHz) 2. 7 2. 4 2. 1 1. 8 1. 5 1. 2 Voltage (V) 0. 99 0. 96 0. 94 0. 92 0. 90 0. 88 Active Power (W) 3. 42 2. 93 2. 49 2. 05 1. 68 1. 31 Limited room for down scaling. 56% frequency reduction 13% voltage reduction Limited power saving. 61% power reduction Per-core control is not common. Sleep States State transition time Target Residency Power C 0 N/A (3~3. 5 W) C 1 1 μs 1. 2 W C 3 59 μs 156 μs 0. 13 W C 6 89 μs 300 μs 0 W Limited by the length of idle periods.

Observations Short idle periods cause high idle power. Traffic variability. Fine-grained control over time

Observations Short idle periods cause high idle power. Traffic variability. Fine-grained control over time and space domain. Our Solution: Dyn. Sleep

Dyn. Sleep: Overview Utilizing per-core sleep state. (space domain) Postponing the request service. Transform

Dyn. Sleep: Overview Utilizing per-core sleep state. (space domain) Postponing the request service. Transform scattered idle periods into a longer one for deep sleep state. (reduce idle power) Dynamically determine core wake-up time. Satisfy the target tail latency constraint. (time domain)

Dyn. Sleep: Example at t=A 321 t=0 WW 31 Target Tail Latencytime Target Tail

Dyn. Sleep: Example at t=A 321 t=0 WW 31 Target Tail Latencytime Target Tail Latency R 1 arrives R 2 arrives R 3 arrives

Dyn. Sleep: Power consumption behavior Power Baseline Dyn. Sleep Active Shallow sleep Deep sleep

Dyn. Sleep: Power consumption behavior Power Baseline Dyn. Sleep Active Shallow sleep Deep sleep Time

Case Study: Memcached Client send requests ∙∙ Clients Libevent fd Read and Parse Data

Case Study: Memcached Client send requests ∙∙ Clients Libevent fd Read and Parse Data ∙ req Worker Threadresult Request Processing Worker Thread Send Response Memcached Server libevent monitors network sockets through epoll for the request arrivals. Independent threads and requests.

Memcached with Dyn. Sleep Libevent Thread l na Dyn. Sleep Calculator W sig Thread

Memcached with Dyn. Sleep Libevent Thread l na Dyn. Sleep Calculator W sig Thread Communication up e ak l na p ee Register/Update Timer sig fd 2 fd 1 fd Libevent Sl Clients Dyn. Sleep Manager fd Read and Parse Data req Request Processing result Send Response Request Processing Thread Two separate threads. A core is woken up by wakeup signal.

Evaluation: Experiment Setup A client server and a request processing server connected over 10

Evaluation: Experiment Setup A client server and a request processing server connected over 10 G Ethernet. Intel Xeon E 5 2697 -V 2 12 -core processor. Only support per-core DFS. On-chip energy sensors with 1 KHz sampling rate.

Evaluation: Power Saving EP Power Saving 80% Dyn. Sleep DVFS DFS 40% 50% Load

Evaluation: Power Saving EP Power Saving 80% Dyn. Sleep DVFS DFS 40% 50% Load 60% 40% 20% 0% 10% 20% 30% 70% 80% At low to medium load, lager latency slack leads to high power saving of Dyn. Sleep. At high load, Dyn. Sleep power saving is comparative to DVFS scheme. Dyn. Sleep significantly outperforms per-core DFS scheme.

Evaluation: Latency Distribution Baseline 95 th: 187µs DFS Dyn. Sleep Target : 686µs DFS

Evaluation: Latency Distribution Baseline 95 th: 187µs DFS Dyn. Sleep Target : 686µs DFS 95 th: 448µs 100% Load 0. 3 CDF 80% Dyn. Sleep 95 th: 665µs 60% 40% 20% 0% 0 200 400 Latency (µsec) 600 800 Baseline has about 500 µsec latency slack. Large gap still left in DFS(DVFS) scheme because of the limited VF down scaling. Dyn. Sleep effective close the gap by postponing request processing.

Load Tail Latency(µsec) Evaluation: Load Changes target tail latency Dyn. Sleep DFS 800 600

Load Tail Latency(µsec) Evaluation: Load Changes target tail latency Dyn. Sleep DFS 800 600 400 200 0 0, 8 0, 6 0, 4 0, 2 0 0 0, 5 1 1, 5 2 2, 5 3 3, 5 4 Time (s) At low load, DFS can’t fully utilize the latency slack. At high load, DFS lacks the responsiveness and frequently violate the constraint. Dyn. Sleep responds instantaneously to the load changes because of the request level updates.

Conclusion Major source of the energy inefficiency comes from the idle power. Non-deterministic and

Conclusion Major source of the energy inefficiency comes from the idle power. Non-deterministic and short idle periods. We propose Dyn. Sleep Reshape the idle periods pattern. Utilize deep sleep states. Dynamically wake up to meet the strict Qo. S constraint. Our memcached prototype demonstrates up to 65% power saving.