Power Management in Data Centers Theory Practice Mor

Power Management in Data Centers: Theory & Practice Mor Harchol-Balter Computer Science Dept Carnegie Mellon University Anshul Gandhi, Sherwin Doroudi, Alan Scheller-Wolf, Mike Kozuch 1

Power is Expensive Annual U. S. data center energy consumption 100 Billion k. Wh or 7. 4 Billion dollars Electricity consumed by 9 million homes As much CO 2 as all of Argentina Sadly, most of this energy is wasted [energystar. gov], [Mc. Kinsey & Co. ], [Gartner] 2

Most Power is Wasted Servers only busy 5 -30% time on average, but they’re left ON, wasting power. [Gartner Report] [NYTimes] q BUSY server: q IDLE server: q OFF server: Load Setup time 260 s 200 W 200 Watts 140 Watts ? Don’t know future load Intel Xeon E 5520 2 quad-core 2. 27 GHz 16 GB memory ALWAYS ON: Provision for Peak Time 3

Talk Thesis Power, P Response Time, T Key Setup Cost ON/OFF ALWAYS ON - High response time + Might save power + Low response time - Wastes power ? x x x x xx x ? 4

Outline q Part I: Theory – M/M/k Understanding the challenges: -- What is the effect of setup time? q Part II: Systems Implementation Dynamic power management in practice 5

M/M/1/Setup Server turns off when idle. m l jobs/sec Setup ~ Exp(a) # busy servers # jobs in system 0, 0 l 0, 1 a m 1, 1 l l m 0, 2 a 1, 2 l l m 0, 3 a 1, 3 [Welch ’ 64] 6

M/M/k/Setup (k=4) # jobs in system 0, 0 l # busy servers m 0, 1 a 1, 1 l l m 2 m m l Setup ~ Exp(a) m m l 0, 2 2 a l 1, 2 a m l 2, 2 2 m 3 m l 0, 3 3 a l 1, 3 m 2 a 2, 3 a 3, 3 l 2 m l 3 m 4 m 0, 4 4 a 1, 4 3 a 2, 4 2 a 3, 4 a 4, 4 l l m l 2 m l 3 m l 4 m 0, 5 4 a 1, 5 3 a 2, 5 2 a 3, 5 a 4, 5 m 7

M/M/k/Setup (k=4) # jobs in system 0, 0 l # busy servers m 0, 1 a 1, 1 l l m 2 m m l Setup ~ Exp(a) m m m l 0, 2 2 a l 1, 2 a m l 2, 2 2 m 3 m l 0, 3 3 a l 1, 3 m 2 a 2, 3 a 3, 3 l 2 m l 3 m 4 m 0, 4 4 a 1, 4 3 a 2, 4 2 a 3, 4 a 4, 4 l l m l 2 m l 3 m l 4 m 0, 5 4 a 1, 5 3 a 2, 5 2 a 3, 5 a 4, 5 Open for 50 years 8

M/M/k/Setup (k=4) # jobs in system 0, 0 l # busy servers m 0, 1 a 1, 1 l l m 2 m m l Setup ~ Exp(a) m m m l 0, 2 2 a l 1, 2 a m l 2, 2 2 m 3 m l 0, 3 3 a l 1, 3 m 2 a 2, 3 a 3, 3 l 2 m l 3 m 4 m 0, 4 4 a 1, 4 3 a 2, 4 2 a 3, 4 a 4, 4 l l m l 2 m l 3 m l 4 m 0, 5 4 a 1, 5 3 a 2, 5 2 a 3, 5 a 4, 5 Solvable only Numerically Matrix-Analytic (MA)

M/M/k/Setup (k=4) # jobs in system 0, 0 l # busy servers m 0, 1 a 1, 1 l l m 2 m m l Setup ~ Exp(a) m m m l 0, 2 2 a l 1, 2 a m l 2, 2 2 m 3 m l 0, 3 3 a l 1, 3 m 2 a 2, 3 a 3, 3 l 2 m l 3 m 4 m 0, 4 4 a 1, 4 3 a 2, 4 2 a 3, 4 a 4, 4 l 0, 5 4 a l m l 2 m l 1, 5 3 a 2, 5 3 m l 4 m Wide Open 2 a 3, 5 a 4, 5

New Technique: RRR [Sigmetrics 13] Finite portion Infinite repeating portion Recursive Renewal Reward (RRR) Ø Exact. No iteration. No infinite sums. Ø Yields transforms of response time & power. Closed-form for all chains that are skip-free in horizontal direction and DAG in vertical direction. 11

New Technique: RRR [Sigmetrics 13] Finite portion Infinite repeating portion N = # jobs 12

m ON/OFF m OPT (ON/OFF with 0 setup) m k E[P] (k. W) l E[T] (s) Results of Analysis ON/OFF OPT 13 k

Outline q Part I: Theory – M/M/k What is the effect of setup time? -- Setup hurts a lot when k: small -- But setup much less painful when k: large -- ON/OFF allows us to achieve near optimal power Demand q Part II: Systems Implementation Dynamic power management in practice ? Time -- Arrivals: NOT Poisson Very unpredictable! -- Servers are time-sharing -- Job sizes highly variable -- Metric: T 95 < 500 ms -- Setup time = 260 s 14

Our Data Center Unknown Power Aware Load Balancer q key-value workload mix of CPU & I/O q 1 Request = 120 ms, 3000 KV pairs q Provision capacity to meet SLA q SLA: T 95 < 500 ms q Setup time: 260 s 500 GB DB 7 Memcached 28 Application servers 15

Provisioning At Single Server T 95 1000 800 600 400 450 ms 400 200 00 10 10 30 30 50 50 70 70 90 90 110 130 arrival rate 60 req/s Always. On r(t) ? t ON/OFF x r(t) x x xxx x x xx ? t 16

Always. On ______ load o kbusy+idle x kbusy+idle+setup Num. servers ______ load o kbusy+idle x kbusy+idle+setup ON/OFF Time (min) T 95=291 ms, Pavg=2, 323 W T 95=11, 003 ms, Pavg=1, 281 W I’m late, I’m late! 17

ON/OFF Variants Reactive Control-Theoretic Predictive [Krioukov, …, Culler, Katz ‘ 10] [Castellanos et al. ‘ 05] [Horvath, Skadron ’ 08] [Urgaonkar, Chandra ‘ 05] [Leite, Kusic, Mosse ‘ 10] [Bennani, Menasce ‘ 05] [Nathuji, Kansal, Ghaffarkhah ‘ 10] [Chen, He, …, Zhao ’ 08] [Gmach et al. ‘ 08] [Fan, Weber, Barroso ‘ 07] [Wang, Chen ‘ 08] [Wood, Shenoy, … ‘ 07] [Chen, Das, …, Gautam ’ 05] [Bobroff, Kuchut, Beaty ‘ 07] -- All suffer from setup lag. -- All too quick to shut servers off. -- All provision based on arrival rate. 18

ON/OFF+padding Num. servers ______ load o nbusy+idle x nbusy+idle+setup Num. servers ON/OFF x = 100 Time (min) T 95=11, 003 ms, Pavg=1, 281 W T 95=487 ms, Time (min) Pavg=2, 218 W 19

A Better Idea: Auto. Scale Two new ideas Existing ON/OFF policies are too quick to turn servers off … then suffer huge setup lag. [Transactions on Computer Systems 2012] Wait some time (twait) before turning idle server off “Un-balance” load: Pack jobs onto as few servers as possible w/o violating SLAs 20

Scaling Up via Auto. Scale When scaling up capacity, # jobs/server is a more robust indicator than current request rate. But not so obvious how to use # jobs/server … 10 jobs/server load r 60 req/s 30 jobs/server load 2 r 120 req/s 2 r r [Transactions on Computer Systems 2012] 21

Why Auto. Scale works Theorem As k → with ∞, Auto. Scale’s capacity Theorem : As(loosely): k → ∞, M/M/k Delayed. Off + Packing approaches square-root staffing. [ Performance Evaluation, 2010 (b)] 22

Auto. Scale Num. servers ______ load o kbusy+idle x kbusy+idle+setup ______ load o nbusy+idle x nbusy+idle+setup Num. servers ON/OFF Time (min) T 95=11, 003 ms, Pavg=1, 281 W I’m late, I’m late! T 95=491 ms, Pavg=1, 297 W Within 30% of OPT power on all our traces! Facebook cluster-testing AS 23

Results Always. On ON/OFF Auto. Scale T 95 Pavg 291 ms 2323 W 11, 003 ms 1281 W 491 ms 1297 W 271 ms 2205 W 3, 802 ms 759 W 466 ms 1016 W 289 ms 2363 W 4, 227 ms 1, 391 W 470 ms 1679 W 377 ms 2263 W > 1 min 849 W 556 ms 1412 W [Transactions on Computer Systems 2012] 24

Conclusion Dynamic power management Managing the setup cost Part I: Effect of setup in M/M/k q First analysis of M/M/k/setup and M/M/∞/setup q Introduced RRR technique for analyzing repeating Markov chains q Effect of setup cost is very high for small k, but diminishes as k increases Part II: Managing the setup cost in data centers q q q Non-Poisson arrival process; load unknown; unpredictable spikes Leaving servers Always. On wastes power, but setup can be deadly. Lesson: Don’t want to rush to turn servers off. Proposed Auto. Scale with Delayedoff, Packing routing & Non-linear Scaling. Demonstrated effectiveness of Auto. Scale in practice. Proved Auto. Scale has very good asymptotic behavior. 25

References Anshul Gandhi, Sherwin Doroudi, Mor Harchol-Balter, Alan Scheller-Wolf. "Exact Analysis of the M/M/k/setup Class of Markov Chains via Recursive Renewal Reward. " Proceedings of ACM SIGMETRICS 2013 Conference, June 2013. Anshul Gandhi, Mor Harchol-Balter, Ram Raghunathan, and Mike Kozuch. "Auto. Scale: Dynamic, Robust Capacity Management for Multi-Tier Data Centers. " ACM Transactions on Computer Systems, vol. 30, No. 4, Article 14, 2012, pp. 1 -26. Anshul Gandhi, Mor Harchol-Balter, and Ivo Adan. "Server farms with setup costs. " Performance Evaluation, vol. 67, no. 11, 2010, pp. 1123 -1138. Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, and Michael Kozuch. "Optimality Analysis of Energy-Performance Trade-off for Server Farm Management. " Performance Evaluation vol. 67, no. 11, 2010, pp. 1155 -1171. Thank You! 26