Virtual Devices for Virtual Machines or Building Extensible
Virtual Devices for Virtual Machines or Building Extensible Devices with a Chainsaw and some Duct Tape Andrew Warfield andrew. warfield@cl. cam. ac. uk
Take-home message: Despite being a coarse-grained tool, VMMs are a great substrate for low-level extensibility. “VMMs: They’re boxy, but they’re good!”
Why do we want to extend devices? 1. Add new device-level systems features. • Versioning storage, encryption, checksum offload. 2. Manage I/O for VMs in a cluster. • • They move around! I/O looks a lot like a “service”. Physical Device (e. g. disk) Extension (e. g. encryption) Soft Device VM (OS + Apps)
What are the challenges: 1. Performance. • “Fast enough to be useful. ” 2. Safety. • Extensions shouldn’t cause system to melt. 3. “Software Engineering”. • Extension writers shouldn’t have to piece together the internals of an OS, learn undocumented interfaces and locking disciplines, or spend all their time tracking versions. Extensions should be portable across OSes.
The rest of this talk: 4 Why VMMs are a good platform for device extensions. 4 a. k. a. how devices work in Xen. 4 An extension interface for Xen. 4 Case: Packet symmetry enforcement. 4 Case: The Parallax storage service. 4 New and somewhat crazy stuff. 4 Conclusion.
Devices in Xen. Problem 2: 1: Stability. Maintenance. VMM is are Drivers nowainhuge the “driver trap”… has amount of (often to support buggy) new hardware, code. We is don’t constantly want to chasing count ondriver them evolution. for reliability. Web Mail OS OS Sometimes they crash. DD VD. D VMM HW Option 1: Switch from VMM single runs. OS physical to VMM: device Web driver. and mail VM servers drivers for are now isolated “virtual” device. (or Either consolidated), real (emulated) but how. HW, to share or idealized. devices? VD. D
A digression… Chainsaw: (Somewhat imprecise surgical instrument) 4 The OS is a component. 4 OSes work, have lots of tools, and know how to talk to other OSes. 4 Maintained by other people. Duct Tape: (Simple yet effective connective mechanism. ) 4 No generalized IPC in the VMM 4 Event notification (virtual irqs) 4 Memory sharing primitives
Devices in Xen. Web OS (Device VM) VD. S DD Mail OS OS VD. D VMM HW Option 2: VMM exports physical hardware to a device VM. Use OS driver, OS mechanisms (e. g. packet forwarding) VD. D
Devices in Xen. 1. Coarse-grained sharing 2. Simple one-bit notification Mail OS (Device VM) OS RRR DD VD. S VD. D VMM HW Option 2: VMM exports physical hardware to a device VM. Use OS driver, OS mechanisms (e. g. packet forwarding)
Devices in Xen. 1. Coarse-grained sharing 2. Simple one-bit notification OS (Device VM) DD Mail R R R OS R VD. S VD. D VMM HW Option 2: VMM exports physical hardware to a device VM. Use OS driver, OS mechanisms (e. g. packet forwarding)
Devices in Xen. 1. Coarse-grained sharing 2. Simple one-bit notification Mail Point of discussing that in (some) detail : OS OS R R (Device VM) • There is a fast, general mechanism for R moving requests around. • Classes of drivers use simple protocols for requests. • VMM is largely in the details. DD uninvolved VD. S • Device driver is an extension. VMM HW Option 2: VMM exports physical hardware to a device VM. Use OS driver, OS mechanisms (e. g. packet forwarding) VD. D
Extending Devices Logging. e. g. debug/analysis Redirecting. e. g. add versioning Rewriting. encryption Extension App. OS (Device VM) OS (Extention VM) Mail OS Dev. Tap DD VD. S VD. D VMM HW The device tap acts as a switch. In addition to interposing All messages, requests may be terminated or passed through.
Devices in Xen. Mail OS (Device VM) DD OS VD. D VMM HW
Extending Devices Extension App. OS (Device VM) OS (Extention VM) Dev. Tap DD VD. S Logging. e. g. debug/analysis Redirecting. e. g. add versioning Rewriting. encryption Extension Goals: Mail • Safety. OS • Software Engineering. • Performance. VD. D VMM HW Trading The device isolation tap acts foras performance. a switch. In addition We may to move interposing the tap All messages, And extension requests code ontomay the be device terminated VM. or passed through.
Extension Case Study: Packet Symmetry 4 Do. S is a big concern in virtual hosting facilities. 4 Well-connected servers, paying for bandwidth. 4 exploited into a botnet == not good! 4 But provider can offer protection: 4 Enforce a symmetry on RX: TX packet counts. 4 See our Hot. Nets paper (Kreibich et al) 4 Interesting extension bits: 4 Extension written in python. 4 Never drops packets.
Extension Case Study: Parallax 4 Managing storage in VM clusters. 4 Storage service “owns” local disks, and the OSes that manage them. 4 Parallax virtualizes storage, fast snapshots, etc. 4 Initially proposed as Hot. OS’ 05 paper. 4 Ongoing, with Net. App and Google. 4 Local disks as persistent caches. 4 Candidate selection for migration. 4 Support explicit snapshot request from OSs. 4 Not using a big filer. *
Pre-conclusion: Take-home message: Despite being a coarse-grained tool, VMMs are a great substrate for low-level extensibility. …now the fun new part. . .
Full-system extensions. I/O Taint Protected VM VN VD Qemu* Control VM ND DD Protected VM VN VD Taint Pagemap VMM Net Disk 4. Taint 1. 2. 3. Inbound VM traps runs markings pages in onemulation, access are marked propagated to tracking a tainted as tainted to page. disk. data. Tainted Fine-grained Disk extension Qemu pagestaint Detailstainted Marked microcode marks innot-present. extension, modified data, and to page-granularity Throw reflect re-taints VM tainting to memory emulation. across bitmap on data read. in VMM. movement.
Full-system extensions. I/O Taint Protected VM VN VD Qemu* Control VM ND DD Protected VM VN VD Taint Pagemap VMM Net Disk 4. Taint 1. 2. 3. Inbound VM traps runs markings pages in onemulation, access are marked propagated to tracking a tainted as tainted to page. disk. data. Tainted Fine-grained Disk extension Qemu pagestaint Detailstainted Marked microcode marks innot-present. extension, modified data, and to page-granularity Throw reflect re-taints VM tainting to memory emulation. across bitmap on data read. in VMM. movement.
Conclusion: Despite being a coarse-grained tool, VMMs are a great substrate for low-level extensibility. 4 VMM is a good base for device extensions. 4 With device tap, can write extensions in userspace, in any OS, without messing with individual kernels. 4 Treating devices as services seems to be a good approach to managing I/O in clusters. 4 V 2 E/E 2 V is very cool.
Fin.
- Slides: 21