Xen and the Art of Virtualization Ian Pratt

Xen and the Art of Virtualization Ian Pratt University of Cambridge and Founder of Xen. Source Inc. Computer Laboratory

Outline ¾Virtualization overview ¾Xen Today : 2. 0 Overview ¾Architecture ¾Performance ¾Live VM Relocation ¾Xen 3. 0 Roadmap (Q 2 2005)

Virtualization Overview ¾Single OS image: Ensim, Vservers, CKRM § Group user processes into resource containers § Hard to get strong isolation ¾ Full virtualization: VMware, Virtual. PC § Run multiple unmodified guest OSes § Hard to efficiently virtualize x 86 ¾Para-virtualization: UML, Xen § Run multiple guest OSes ported to special arch § Arch Xen/x 86 is very close to normal x 86

Xen Today : 2. 0 Features ¾Secure isolation between VMs ¾Resource control and Qo. S ¾Only guest kernel needs to be ported § All user-level apps and libraries run unmodified § Linux 2. 4/2. 6, Net. BSD, Free. BSD, Plan 9 ¾Execution performance is close to native ¾Supports the same hardware as Linux x 86 ¾Live Relocation of VMs between Xen nodes

Para-Virtualization in Xen ¾Arch xen/x 86 : like x 86, but replaces privileged instructions with Xen hypercalls § Avoids binary rewriting and fault trapping § For Linux 2. 6, only arch-dep files modified ¾Modify OS to understand virtualised env. § Wall-clock time vs. virtual processor time • Xen provides both types of alarm timer § Expose real resource availability • Enables OS to optimise behaviour ¾MMU virtualisation: direct vs. shadow mode

I/O Architecture ¾Xen IO-Spaces delegate guest OSes protected access to specified h/w devices § Virtual PCI configuration space § Virtual interrupts ¾Devices are virtualised and exported to other VMs via Device Channels § Safe asynchronous shared memory transport § ‘Backend’ drivers export to ‘frontend’ drivers § Net: use normal bridging, routing, iptables § Block: export any blk dev e. g. sda 4, loop 0, vg 3

Xen 2. 0 Architecture VM 0 VM 1 VM 2 VM 3 Device Manager & Control s/w Unmodified User Software Guest. OS (Xen. Linux) (Xen. BSD) Back-End Front-End Device Drivers Native Device Driver Control IF Native Device Driver Safe HW IF Event Channel Virtual CPU Virtual MMU Xen Virtual Machine Monitor Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)

System Performance 1. 1 1. 0 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0. 0 L X V U SPEC INT 2000 (score) L X V U Linux build time (s) L X V U OSDB-OLTP (tup/s) L X V U SPEC WEB 99 (score) Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)

TCP results 1. 1 1. 0 0. 9 0. 8 0. 7 0. 6 0. 5 0. 4 0. 3 0. 2 0. 1 0. 0 L X V U Tx, MTU 1500 (Mbps) L X V U Rx, MTU 1500 (Mbps) L X V U Tx, MTU 500 (Mbps) L X V U Rx, MTU 500 (Mbps) TCP bandwidth on Linux (L), Xen (X), VMWare Workstation (V), and UML (U)

Scalability 1000 800 600 400 200 0 L X 2 L X 4 L X 8 L X 16 Simultaneous SPEC WEB 99 Instances on Linux (L) and Xen(X)

Live VM Relocation ¾Why is VM relocation useful? § Managing a pool of VMs running on a cluster § Taking nodes down for maintenance § Load balancing VMs across the cluster ¾Why is it a challenge? § VMs have lots of state § Some VMs will have soft real-time requirements • E. g. web servers, databases, game servers § Can only commit limited resources to migration

Rate Limited Migration

Quake 3 Server migration

Roadmap for Xen 3. 0 ¾SMP guest OSes § Prototype now working, undergoing tuning § Required careful design to get good performance and retain security guarantees ¾Support for Intel VT-x extensions § Run ‘legacy’ unmodified OSes ¾Other ports : x 86/64 and ia 64 (ppc) § Both Xen x 86/64 and ia 64 now boot!

Roadmap for Xen 3. 0 ¾Better cluster management tools § Manage pool of VMs across a set of nodes ¾Better tools for Qo. S control ¾New GUI management tool ¾Improved hardware compatibility § Graphics cards, ACPI, APM

Research Roadmap : 4. 0 ¾ Cluster load balancing algorithms § Exploit properties of live migration ¾ Software fault tolerance § Exploit deterministic replay ¾ System debugging § Lightweight checkpointing and replay ¾ VM forking § Lightweight service replication, isolation ¾ Secure virtualization § Multi-level secure Xen

Conclusions ¾Xen is a complete and robust GPL VMM ¾Outstanding performance and scalability ¾Excellent resource control and protection ¾Live relocation makes seamless migration possible for many real-time workloads ¾http: //xensource. com ¾http: //xen. sf. net