Improving Data Center Efficiency Andrew Kutz Improving Data

Improving Data Center Efficiency Andrew Kutz

Improving Data Center Efficiency Thesis ◦ The energy efficiency of today’s information technology (IT) infrastructure is not doubling every two years along with performance, leading to very high performing, very inefficient data centers. The inefficiencies in energy consumption must be mitigated so that efficiency will catch up with performance. 2

Improving Data Center Efficiency Agenda ◦ Power in the data center today ◦ Areas for improvement ◦ Recommendations 3

Improving Data Center Efficiency Agenda ◦ Power in the data center today �Data center power distribution �Data center power consumption �The result ◦ Areas for improvement ◦ Recommendations 4

Power in the data center today Data center power distribution Power Station Backup Generator(s) PSU UPS Compute Servers Power distribution units Racks Room, misc. Storage CPU HVAC Storage devices 5

Power in the data center today Data center power consumption Data center 40 % 30% IT equipment Power delivery Cooling equipment IT equipment 60% 20% Compute servers Storage servers Comm. & misc. Server components 35% 15% 15% 20% Source: Dell, APC, SUN ◦ The breakdown leaves room for plenty of areas of improvement. CPUs CPU VR loss Memory HDDs and misc. PSU loss 6

Power in the data center today The result 75 x 16 x Source: Christian Belady 7

Improving Data Center Efficiency Agenda ◦ Power in the data center today ◦ Areas for improvement �Processors �Power supplies �Power delivery �Cooling ◦ Recommendations 8

Areas For Improvement Processors – Last gen processors were power hungry beasts! ◦ A first-generation AMD Opteron (856) at 3. 0 GHz used 92. 6 W ◦ According to AMD, processors can use between 5060% of rated power when in an idle state. ◦ That means the same processor was consuming 46. 3 - 55. 56 W of energy when doing nothing. 9

Areas For Improvement Processors – Multi-core CPUs ◦ Multiple cores on a single chip and processor development helps to reduce energy consumption �An Intel Xeon 3. 80 GHz used 110 W �An Intel Xeon E 5345 2. 33 GHz Quad-Core uses 80 W. � 2. 45 times the processing power at. 72 times the energy consumption! 10

Areas For Improvement Processors – Power stepping CPUs ◦ CPUs can step down into reduced performance modes by adjusting frequency and voltage in synchronization with load. Source: AMD 11

Areas For Improvement Power supplies – Today’s problems … ◦ The typical efficiency of today’s power supply is anywhere between 75 -80%. ◦ 1000 servers @ 400 W @ 75% efficiency means 100 k. W is being radiated into heat that cooling systems must remove. ◦ 100 k. W equals 341, 400 BTU. ◦ At 0. 10 USD/ k. Wh it costs 52, 317. 07 USDover a 4 year life cycle to remove the extra heat. 12

Areas For Improvement Power supplies – Are tomorrow’s opportunities ◦ Power supplies being developed with digital feedback control can maintain 90 -95% efficiency over a wide range. ◦ 1000 servers @ 400 W @ 95% efficiency means only 20 k. W, or 68, 280 BTU, is radiated as heat. ◦ At 0. 10 USD / k. Wh it costs 14, 953. 32 USD over a four year life cycle to remove the extra heat. ◦ Increasing PSU efficiency by 20% reduces wasted cooling costs by four fifths. 13

Areas For Improvement Power delivery – Uninterruptible Power Supplies (UPS) ◦ UPS devices are long-term investments. However, it may be time to upgrade. Source: Eaton Power ◦ May lose. 59% efficiency for every year a UPS device is in service. That is a potential improvement of 10. 03% for upgrading a UPS from 1990. 14

Areas For Improvement Cooling – Inefficiencies ◦ General cooling is a guaranteed way to generate hot spots, resulting in wasted energy. ◦ Over-cooling – Cooling systems are often designed to run at maximum capacity at all times, resulting in wasted energy. 15

Areas For Improvement Cooling – Improving Efficiency ◦ Targeted cooling and right-sizing �The closer the cooling is to the source of the heat, the better. �While it is tempting to buy big, only so much cooling is needed. Any more than is needed is wasted energy. �Take off the sweaters! ◦ Remove the … �The heat – the biggest barrier to cooling is heat! �The gaps – seal cable cut-outs and install blanking panels. ◦ Investigate liquid cooling �Or how I learned to stop worrying and learned to love the H 2 O. �APC, IBM, Liebert are offering liquid cooled rack solutions. 16

Areas for Improvement Direct Current (DC) Power Delivery ◦ Pros �Delivering DC power directly may offer greater energy efficiency by reducing the number of distinct AC DC inductors. �One side effect would be reducing the amount of cooling required per server since excess heat in the AC DC conversion is localized to fewer areas. ◦ Cons �Only one major vendor (Rackable Systems) currently offers full DC support in its x 86 server product line. This could lead to vendor lock-in. �Although used for years by the Tel. Co industry, the number of engineers experienced in DC power vs. AC power is small. 17

Areas For Improvement Cooling – Improving efficiency (continued) ◦ Variable Frequency Drives (VFDs) �A subset of adjustable speed drives (ASDs). �Adjusts speed by altering the frequency and voltage output. �Cooling equipment with VFDs can dynamically adjust the speed of the fan based on different input factors such as static pressure using a pitot tube and manometer. �Reducing the output voltage when extra cooling is not need leads to less energy consumption. 18

Improving Data Center Efficiency Agenda ◦ Power in the data center today ◦ Areas for improvement ◦ Recommendations �Technical �Business �Community 19

Recommendations Technical ◦ Remember, CPUs represent almost 10% of a data center’s power consumption. ◦ Rethink power and cooling designs and systems. ◦ Explore consolidation efforts: �Compute �Virtualization �Blades �Storage �Shared storage – SANs, Filers �Storage designs – massive array of idle disks (MAID), data deduplication 20

Recommendations Technical ◦ Delivering DC power directly to the servers sounds like a good idea, but it should be classified under “wait and see. ” 21

Recommendations Business ◦ Make the power and cooling issue a bold line item on the Cx. O’s budget. ◦ Consider creative options: �Rent roof space to local utilities companies for solar panels and buy back cheap energy. �Move the data center to a colder climate. ◦ Pressure vendors to think green. Purchasing power is the greatest bargaining tool that customers have. ◦ Every percent counts. A 400 k. W data center @ 0. 10 USD / k. W = 245, 173. 46 USD / year. 1% will result in an annual savings of 3, 504 USD. 15% = 36, 775. 77 USD. 30% = 73, 551. 54 USD. 22

Recommendations Community ◦ Join organizations, such as the green grid, that are dedicated to helping to build an energy efficient data center. ◦ Pay attention to existing and emerging standards like Green Grid’s Power Usage Effectiveness (PUE) when buying new equipment to ensure energy efficiency. ◦ The Energy Star is expanding to cover servers. ◦ Encourage vendors who are already thinking green to keep it up: Fujitsu-Siemens, NEC, IBM, HP, Dell, and Sun to name a few. 23

Conclusions ◦ The opportunity to increase efficiency will probably not disappear while electricity prices are not prohibitively expensive and compute power is cheap. ◦ The government and industry initiatives are developing and will continue to fine tune energy efficiency related standards. Efficiency used to be Power Out / Power In. It will be transformed to Productivity Out / Power In. ◦ To have the most effect, attack the largest culprit first – processors. CPUs account for almost 10% of a data center’s energy consumption. Save the power leader, save the world! 24

References ◦ AMD – http: //enterprise. amd. com/Downloads/34146 D_ PC_WP. pdf ◦ APC – http: //www. apcmedia. com/salestools/NRAN 6 CN 8 PK_R 0_EN. pdf ◦ Christian Belady – http: //electronicscooling. com/articles/2007/feb/a 3/ ◦ Dell – http: //www. dell. com/downloads/global/power/p s 1 q 07 -20070210 -Cover. Story. pdf ◦ Intel – http: //www. intel. com 25