SESSION CODE WSV 316 Robert Larson Delivery Architect

SESSION CODE: WSV 316 Robert Larson Delivery Architect Microsoft Corporation David Lef Principal Systems Architect Microsoft Corporation

Session Objectives:

Windows Server 2008 R 2 Scalable Platform Enhancements i. SCSI Breakthrough Performance Results Storage: Hyper-V Options and Best Practices A Real World Deployment: Microsoft IT’s Server Environment MSIT’s Hyper-V Deployment MSIT’s “Scale Unit” Virtualization Infrastructure Questions and Answers

Windows Server 2008 R 2 Scalable Platform: Efficient scaling across multi-core CPUs Compute • 256 processor core support • Core Parking • Advanced Power Management Storage • 256 processor core IO scaling • Dynamic Memory allocation • i. SCSI Multi-Core scaling (NUMA IO) • Networking • 256 processor core support • NUMA awareness • VMQ and Virtualization performance Virtualization Hyper-V™ • Hot Add Storage • Intel EPT memory management support • Live Migration Intel® Ethernet Adapters Intel® Xeon® Processor

Extending the i. SCSI platform i. SCSI and Storage Enhancements in R 2 Reliability, Scalability and Performance • i. SCSI Multi-Core and Numa IO • DPC redirection • Dynamic Load Balancing • Storage IO Monitoring • CRC Digest Offload • Support for 32 paths at boot time PHYSICAL Management • i. SCSI Quick Connect • Configuration Reporting • Automated deployment • i. SCSI Server Core UI VIRTUAL

Windows Server 2008 R 2 Scalable Platform Enhancements i. SCSI Breakthrough Performance Results Storage: Hyper-V Options and Best Practices A Real World Deployment: Microsoft IT’s Server Environment MSIT’s Hyper-V Deployment MSIT’s “Scale Unit” Virtualization Infrastructure Questions and Answers

Filesystem Provided by: Volume Manager Microsoft Class/Disk Firmware/ Driver Hardware Port Driver (Storport) IHV MSISCSI. SYS OS TCPIP. SYS HW OS TCP Chimney HW RSS i. SCSI HBA miniport NDIS MINIPORT LRO LSO MSISCSI. SYS RSS LRO LSO i. SCSI HBA HW

i. SCSI Test Configuration – 2008 R 2 1 Gbps Switch 1 Gbps Iometer Management System i. SCSI Soft Targets Software Initiator Switch Cisco*Nexus* 5020 (10 Gb. E) Single Port 10 Gb. E Server • Windows Server 2008 R 2 • Microsoft i. SCSI Initiator • Intel ®Xeon® Processor 5580, quad core, dual socket, 3. 2 Ghz, 24 GB DDR 3, MTU 1500, Outstanding I/Os =20 * Other names and brands may be claimed as the property of others. Adapter Target LUN 1 Target LUN 2 Target LUN 3 Target LUN 4 Target LUN 5 Target LUN 6 Target 10 Gbps per Target • Intel® Ethernet Server Adapter X 520 based on Intel® 82599 10 Gb. E Controller. LUN 7 LUN 8 Target LUN 9 Target LUN 10 Performance factors • i. SCSI initiator perf optimizations • Network stack optimizations • Receive Side Scaling (RSS) • Intel Xeon 5500 QPI and integrated memory controller • Intel® 82599: HW Acceleration, multi-core scaling with RSS, MSI-X

Breakthrough Performance at 10 Gb. E Intel® Xeon® Processor 5580 Platform, Windows Server 2008 R 2 and Intel® 82599 10 Gb. E Adapter Read/Write IOPs and Throughput Test 1, 030, 000 IOPs • • • Read/Write IOPs and CPU Test Single Port 10 Gb. E line rate 10 k IOPs per CPU point Performance for real world apps Future ready: Performance Scales 552 k IOPs at 4 k represents • • • 3, 100 Hard Disk Drives 400 x a demanding database workload 1. 7 m Exchange mailboxes 9 x transactions of large e. Tailers Jumbo frames: >30% CPU decrease is common for larger IO size (jumbo frames not used here) Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Microsoft and Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing.

2008 R 2 Hyper-V i. SCSI Test Configuration 1 Gbps Switch i. SCSI Direct connection i. SCSI initiator runs in VM Microsoft VMQ and Intel VMDq Iometer Management System Switch Cisco*Nexus* 5020 (10 Gb. E) 10 Gbps per Target LUN 1 Target LUN 2 Target LUN 3 Target LUN 4 Target LUN 5 Target LUN 6 Target LUN 7 Target LUN 8 Target LUN 9 Target LUN 10 i. SCSI Soft Lab Targets Performance factors Single 10 Gbps Port Host Physical connection Virtual connection * Other names and brands may be claimed as the property of others. • i. SCSI initiator perf optimizations • Microsoft network stack optimizations • Hyper-V scaling • Receive Side Scaling on host • Microsoft VMQ • Intel VMDq

Breakthrough Performance – Hyper-V i. SCSI Performance with Intel® 82599 10 G NIC with VMDq, Intel® Xeon 5580 Platform, Windows Server 2008 R 2 and R 2 Hyper V Read/Write IOPs and Throughput Test • 715 k IOPs -- 10 Gb. E line rate • • • Future ready: Scales with new platforms, OS an Near native i. SCSI performance Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Microsoft Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing.

i. SCSI Performance Test Conclusions i. SCSI protocol performance is only limited by the speed of the underlying bus and vendor implementation § i. SCSI is ready for mission critical performance workloads § Use Receive Side Scaling to optimize for i. SCSI performance in the host § Use VMQ/VMDq to optimize i. SCSI performance within the VM for best IO scaling § VLANs offers logical separation of LAN/SAN traffic and additional performance isolation § Most applications use moderate IO and throughput §

Windows Server 2008 R 2 Scalable Platform Enhancements i. SCSI Breakthrough Performance Results Storage: Hyper-V Options and Best Practices A Real World Deployment: Microsoft IT’s Server Environment MSIT’s Hyper-V Deployment MSIT’s “Scale Unit” Virtualization Infrastructure Questions and Answers

CSV Requirements

Storage Deployment Factors

General Guidance VM 1 VHD VHD VHD VM 2 VM 3 Each VM required 3 separate CSVs However in this case: 3 X 3 = 3

With 16 Cluster nodes sending I/O to single LUN…

Work with your storage provider

CSV and Hyper-V Backup Protect from parent or within the guest VM? Answer – Both! Backup from Hyper-V Parent Partition Protect the virtual machine and associated VHDs Includes non-Windows servers Backup from Guest VM Protect application data SQL database Exchange Share. Point Files Same as protecting a physical server

Clients Windows Server Hosts Fibre Channel switch Network § Increases uptime of Windows Server by providing multiple paths to storage § Increased server bandwidth over multiple ports § Automatic failure detection and failover § Dynamic load balancing § Works with Microsoft DSM provided inbox or 3 rd party DSMs (Power. Path, On. TAP DSM, etc. )

• • • Always use active/active multipathing load balance policies (round robin, least queue depth) vs. failover mode. Follow vendor guidelines for timer settings • Queuedepth, PDORemove. Period, etc. as these settings will have a direct impact on ability to deliver max IO to VMs and controlling failover times Many array vendors include host utilities that automatically adjust settings to optimize for their array • Example: Dell “hit kit” host integration kit Pay attention to spindle count for workload • SSDs (Solid State Drives) change the game on this For NICs used for i. SCSI in HBA mode with offload (i. SOE), turn off TCP Chimney • http: //support. microsoft. com/kb/951037

Windows Server 2008 R 2 Scalable Platform Enhancements i. SCSI Breakthrough Performance Results Storage: Hyper-V Options and Best Practices A Real World Deployment: Microsoft IT’s Server Environment MSIT’s Hyper-V Deployment MSIT’s “Scale Unit” Virtualization Infrastructure Questions and Answers

MSIT Enterprise Virtualization - History

MSIT Enterprise Virtualization - Current

“Discrete Unit” Proliferation

MSIT’s Solution - The “Scale Unit” Virtualization Platform

Scale Unit – Basic Design Elements

Deployment Metrics Example Comparison Point Discrete Units* Scale Unit v 1 Scale Unit v 2 Rack Space (Rack Units) 1408 U 440 U 200 U Rack Space (Number of Racks) 32 Racks 11 Racks 5 Racks Power (Kilowatts) 320 KW 185 KW 90 KW Max Heat (BTU/Hr) 1. 1 Million BTU/Hr 0. 6 Million BTU/Hr 0. 3 Million BTU/Hr Power Cords/Ports 1408 (C 13) 264 (C 19) 120 (C 19) Network Cables/Ports 1408 44 40 Storage Cables/Ports 1408 176 80 Avg CPU Utilization ~15 -20% ~30 -40% ~70 -80% RAM Available 22. 5 TB (32 GB per server) 22. 5 TB (32 GB per blade) 46 TB (144 GB per blade) Avg VMs per Host** 6 6 32 * Equivalent number of traditional 2 U rack-mounted servers

Compute Scale Unit - Physical Layout

Logical Layout Overlay

MSIT R 2 Production Clusters

Storage Scale Unit - History and New Design Principles

Storage Scale Unit - Deployment Details

Physical Layout

Storage Scale Unit - Key Metrics

Tiers of Capacity Determining resource assignment and service levels Tier Description I/O Threshold Price 1 Dedicated space > 5000 IOps > 80 MB/s Straight pass-through + consulting cost (per engagement) 2 Shared R 1 500 -5000 IOps 25 -80 MB/s 3 x shared SU Pricing 3 Shared R 6 < 500 IOps < 25 MB/s Standard SU Pricing

Element Name Provisioning Lock (No new presentations) Grow Lock (No volume growth) Mitigation (Start migrating servers and applications) Pool Oversubscription 150% 200% N/A Pool Written Capacity 70% 80% 90% Avg. Performance Thresholds 60% 70% 80%

Scale Unit - Future Storage Opportunities

Windows Server 2008 R 2 Scalable Platform Enhancements i. SCSI Breakthrough Performance Results Storage: Hyper-V Options and Best Practices A Real World Deployment: Microsoft IT’s Server Environment MSIT’s Hyper-V Deployment MSIT’s “Scale Unit” Virtualization Infrastructure Questions and Answers

www. microsoft. com/teched www. microsoft. com/learning http: //microsoft. com/technet http: //microsoft. com/msdn

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31 st http: //northamerica. msteched. com/registration You can also register at the North America 2011 kiosk located at registration Join us in Atlanta next year