Project Green Light DC Modular Direct Current Datacenters

Project Green. Light - DC Modular Direct Current Datacenters for Improved Energy Efficiency Global Cyber. Bridges 2009 Calit 2 workshop July 1 st, 2009 Dr. Greg Hidley California Institute for Telecommunications and Information Technology, UCSD

IPCC Recommends 25 -40% GHG Reduction Below 1990 Levels by 2020 • On September 27, 2006, Governor Schwarzenegger signed California the Global Warming Solutions Act of 2006 – Assembly Bill 32 (AB 32) – Requires Reduction of GHG by 2020 to 1990 Levels – 15% Reduction from 2008 Levels – 4 Tons of CO 2 -equiv. for Every Person in California • The European Union Requires Reduction of GHG by 2020 to 20% Below 1990 Levels (12/12/2008) • Australia has Pledged to Cut by 2020 its GHG Emissions 5% from 2000 Levels via the Cap &Trade Scheme (12/15/08) • Neither the U. S. or Canada has an Official Target Yet – President Elect Obama Has Endorsed the AB 32 2020 Goal – House Bill passed Friday targeting 17% below 2005 levels and 83% reductions by 2050

ICT is a Critical Element in Achieving Countries Greenhouse Gas Emission Reduction Targets Applications of ICT could enable emissions reductions of 7. 8 Gt CO 2 e in 2020, or 15% of business as usual emissions. But it must keep its own growing footprint in check and overcome a number of hurdles if it expects to deliver on this potential. www. smart 2020. org

The Global ICT Carbon Footprint Roughly the Same as the Aviation Industry Today But ICT is Growing at 6% Annually! www. smart 2020. org

ICT Industry is Already Acting to Reduce Carbon Footprint

Electricity Usage by U. S. Data Centers: Emission Reductions are Underway Source: Silicon Valley Leadership Group Report July 29, 2008 https: //microsite. accenture. com/svlgreport/Documents/pdf/SVLG_Report. pdf

The NSF-Funded Green. Light Project Giving Users Greener Compute and Storage Options UCSD Structural Engineering Dept. Conducted Sun MD Tests May 2007 • Measure and Control Energy Usage: – – – Sun Has Shown up to 40% Reduction in Energy Active Management of Disks, CPUs, etc. Measures Temperature at 5 Levels in 8 Racks Power Utilization in Each of the 8 Racks Chilled Water Cooling Systems $2 M NSF-Funded Green. Light Project UCSD (Calit 2 & SOM) Bought Two Sun MDs May 2008 Source: Tom De. Fanti, Calit 2; Green. Light PI

The Green. Light Project: Instrumenting the Energy Cost of Computational Science • Focus on 5 Communities with At-Scale Computing Needs: – – – Metagenomics Ocean Observing Microscopy Bioinformatics Digital Media • Measure, Monitor, & Web Publish Real-Time Sensor Outputs – Via Service-oriented Architectures – Allow Researchers Anywhere To Study Computing Energy Cost – Enable Scientists To Explore Tactics For Maximizing Work/Watt • Develop Middleware that Automates Optimal Choice of Compute/RAM Power Strategies for Desired Greenness • Partnering With Minority-Serving Institutions Cyberinfrastructure Empowerment Coalition Source: Tom De. Fanti, Calit 2; Green. Light PI

Threat to CI Deployment—CSE Research Needed on How to Deploy a Green CI MRI • Computer Architecture – Rajesh Gupta/CSE • Software Architecture – Amin Vahdat, Ingolf Kruger/CSE • Cine. Grid Exchange – Tom De. Fanti/Calit 2 • Visualization – Falko Kuster/Structural Engineering • Power and Thermal Management – Tajana Rosing/CSE • Analyzing Power Consumption Data – Jim Hollan/Cog Sci http: //greenlight. calit 2. net Source: Tom De. Fanti, Calit 2; Green. Light PI • Direct DC Datacenters – Tom Defanti, Greg Hidley

Green. Light Project: Putting Machines To Sleep Transparently Rajesh Gupta, UCSD CSE; Calit 2 Network interface Management software Main processor, RAM, etc Secondary processor Network interface Low power domain Peripheral Laptop Somniloquy Enables Servers to Enter and Exit Sleep While Maintaining Their Network and Application Level Presence 10

Virtualization at Cluster Level for Consolidation and Energy Efficiency Source: Amin Vadhat, CSE, UCSD Original Service Usher Virtualized Service • Fault Isolation and Software Heterogeneity, Need to Provision for Peak Leads to: – Severe Under-Utilization – Inflexible Configuration – High Energy Utilization • Usher / Die. Cast enable: – Consolidation onto Smaller Footprint of Physical Machines – Factor of 10+ Reduction in Machine Resources and Energy Consumption

UCSD is Installing Zero Carbon Emission Solar and Fuel Cell DC Electricity Generators UCSD 2. 8 Megawatt San Diego’s Point Loma Wastewater Treatment Plant Produces Waste Methane Fuel Cell Power Plant Uses Methane Available Late 2009 2 Megawatts of Solar Power Cells Being Installed

One-line Diagram of Power Distribution

Zero Carbon Green. Light Experiment: Direct DC-Powered Modular Data Center • Concept—Avoid DC to AC to DC Conversion Losses – – Computers Use DC Power Internally DC Fuel Cell 2800 k. Watts Solar and Fuel Cells Produce DC Sun Box <200 k. Watt Both Plug into the AC Power Grid Can We Use DC Directly (With or Without the AC Grid)? • DC Generation Can Be Intermittent – Depends on Source – Solar, Wind, Fuel Cell, Hydro – Can Use Sensors to Shut Down or Sleep Computers – Can Use Virtualization to Halt/Shift Jobs • Experiment Planning Just Starting – Collaboration with Sun and LBNL – NSF Green. Light Year 2 and Year 3 Funds Source: Tom De. Fanti, Calit 2; Green. Light PI

ARRA Funding Opportunity • • Recovery Act: Energy Efficient Information and Communication Technology UCSD Proposal: Renewable, Bio-Fueled 400 VDC Modular Data Center – PI Tom De. Fanti, UCSD, Calit 2 – Co. PIs Rajesh Gupta, Tajana Rosing, Greg Hidley – Partners – NSF funded Green. Light investigators – Lawrence Berkeley National Laboratory – Sun Microsystems, Intel, Delta – EPRI, Direct Power Technologies – Coordination with UCSD Strategic Initiatives Office and Facilities Management • Goals: At UCSD East Campus Utility Plant – – – Establish renewable energy facility with Fuel Cell, Biomass generator, and PVs Develop Smart Grid distribution of above DC sources providing 5 9’s reliability Develop all DC Sun MD with DC powered computers and networking equipment Develop novel cooling improvements using reduced loads and virtualization Develop novel energy efficiency improvements using reduced loads and virtualization

Energy Efficiency Improvements Anticipated • We anticipate energy efficiency improvements as follows – Reductions in all levels of energy chain from FC to servers using 400 VDC – Fuel Cell Inverter (DC to AC) – DC-DC conversion losses (adjust to grid voltage) – 480 VAC distribution line losses – 480 VAC to 208 VAC or 120 VAC step down losses – UPS (AC to DC to charge batteries and back to AC) conversion losses – PDU’s (AC voltage stepped down to server requirements) – Inside Power Supplies (rectification and transformation) – Reduction in cooling loads given heat dissipation of conversions – Anticipate reduction of power utilization by 25% or better (power and cooling) – Anticipate DCi. E of 80% or better, cooling efficiencies of 33% or better – Data Center Infrastructure Efficiency (DCi. E) – DCi. E = IT (useful work) power / total facility power – Power Usage Effectiveness (PUE, the inverse of DCi. E) – PUE = total facility power / IT (useful work) power – DCi. E of 80% is the same as PUE of 1. 25 … implies – 200 kw Sun MD would use 160 kw for computers and 40 kw for cooling and power distribution losses

2. 8 MW FCE Fuel Cell Installation

Toward “Zero Carbon” ICT Green Cloud Computing and Storage • Purchasing Green Power Locally is Expensive with Significant Transmission Line Losses – Demand for Green Power Within Cities is Growing Dramatically – ICT Facilities Don’t Need To Be Located In Cities • But Most Renewable Energy Sites are Very Remote and Impractical to Connect to Electrical Grid – – Can be Easily Reached by an Optical Network Provide Independence from Electrical Utility Savings in Transmission Line Losses (Up To 15% Alone) Plus Carbon Offsets Can Pay for Moving ICT Facilities to Renewable Energy Site • Calit 2 is Discussing Partnering with Canada – Move a Green. Light Facility to Hydro Site in British Columbia, – Link by 10 Gbps Optical Fiber to Calit 2—Offer to Remote Users Source: Bill St. Arnaud, CANARIE, Canada

Coupling AARNet - CENIC/PW - CANARIE Optical Nets: An Australian-U. S. -Canada Green Cloud Testbed Toward Zero Carbon ICT

Application of ICT Can Lead to a 5 -Fold Greater Decrease in GHGs Than its Own Carbon Footprint While the sector plans to significantly step up the energy efficiency of its products and services, ICT’s largest influence will be by enabling energy efficiencies in other sectors, an opportunity that could deliver carbon savings five times larger than the total emissions from the entire ICT sector in 2020. --Smart 2020 Report Major Opportunities for the United States* – – Smart Electrical Grids Smart Transportation Systems Smart Buildings Virtual Meetings * Smart 2020 United States Report Addendum www. smart 2020. org

“It Will Be the Biggest Single Peacetime Project Humankind Will Have Ever Undertaken”