Mobile IO virtualization Harvey Tuch Staff Engineer Mobile

Mobile I/O virtualization Harvey Tuch, Staff Engineer, Mobile Virtualization Platform January 25 th 2012 Sponsored by MIT and VMware Academic Programs VMware: www. vmware. com VMware Labs: labs. vmware. com © 2010 VMware Inc. All rights reserved

Agenda § Smartphone hardware § I/O virtualization techniques § Examples from MVP hypervisor • Storage virtualization • GPS virtualization • Telephony virtualization 2

Smartphone hardware 3

Computer System Organization CPU Memory MMU Controller Local Bus Interface High-Speed I/O Bus NIC Controller Bridge Frame Buffer LAN Low-Speed CD-ROM 4 USB I/O Bus

Evolution of mobile phone to Mobile Personal Computer 5

6 Recent smartphone specs sample (source: pdadb. net) 6 Copyright ® VMware, Inc. All Rights Reserved.

Hardware trends § Example smartphone circa 2012 • • > 1 GHz dual or quad core ARM Cortex-A 9/A 15 or Snapdragon 1 GB SDRAM 512 MB internal NAND Flash 32 GB micro. SD Integrated GPS, > 10 MP camera, 3 G/4 G (LTE), 802. 11, Bluetooth 480 x 800 OLED Ice Cream Sandwich Android OS § Tablets, smartbooks, etc. 7 Copyright ® VMware, Inc. All Rights Reserved.

Smartphone physical hardware • Processor core – ARM • Power efficiency, battery life • E. g. 8 DMIPS/m. W (Cortex-A 9) • ARMv 7 ISA • Devices – highly integrated • So. C organization • Plethora of devices (IP blocks) • Limited standardization • Storage – solid state NAND 8

Agenda § Smartphone hardware § I/O virtualization techniques § Examples from MVP hypervisor • Storage virtualization • GPS virtualization • Telephony virtualization 9

I/O virtualization techniques 10

Device virtualization goals § Present to guest a “virtual” platform (set of devices) • Guest applications, middleware, OS expect standard set of mobile devices • E. g. touchscreen, display, storage, GPS, Bluetooth, Wi. Fi, GSM, GPU, cameras, accelerometers, audio, keyboard, etc. § Multiplex physical hardware between VMs • • • 11 Securely sharing physical hardware across domains Efficiency Fidelity Portability Interposition (checkpointing, live migration) Minimizing engineering effort (cost)

Device virtualization techniques § Device emulation • VMM emulates registers and memory map of physical HW devices § Paravirtualization • VMM provides optimized high-level API for guest devices § Pass-thru • VMM provides guest direct access to physical HW devices 12

Device virtualization techniques – device emulation § VMM emulates registers and memory map of physical HW devices ✓Leverage existing guest OS driver support ✓Interposable ✗Emulation overhead ✗VMM implementation cost Guest Operating System ACME UART driver Write FIFO ‘a’ Read status register CSR Write FIFO ‘b’ …. . ACME UART device model Physical FOOBAR UART Hypervisor Write FIFO ‘a’ Write FIFO ‘b’ Read status register FSR 13 …. .

Device virtualization techniques - paravirtualization § VMM provides optimized high-level API for guest devices ✓Higher performance possible ✓Simplified VMM ✓Interposable ✗Requires custom drivers/libraries per guest Guest Operating System PV UART driver Write shared buffer ‘abcdefghijklmno…’ …. . PV UART device model Physical FOOBAR UART Hypervisor Write FIFO ‘a’ Write FIFO ‘b’ Read status register FSR 14 …. .

Device virtualization techniques – pass-thru § VMM provides guest direct access to physical HW devices ✓Highest performance, even simpler VMM ✗I/O MMU required for safety (DMA danger) ✗Multiplexing HW devices requires extra HW support ✗Interposition hard (hybrid models separate control/data paths) Guest Operating System Write FIFO ‘a’ ACME UART driver Read status register Write FIFO ‘b’ …. . Physical ACME UART Hypervisor 15

Bare-metal (Type-1) hypervisor architecture § Execute directly on physical HW § Preferred architecture for high-end servers § E. g. VMware ESX, Xen, Microsoft Hyper-V 16

Hosted (Type-2) hypervisor architecture § Executes alongside existing host OS, e. g. Mac OS X, Linux, Windows § Leverage host device drivers and resources management § E. g. VMware Workstation/Fusion, KVM, Parallels Desktop 17

1 VMware MVP system architecture § Personal applications run natively § Corporate applications run in a virtual machine 18 Copyright ® VMware, Inc. All Rights Reserved.

1 MVP as a hosted (Type-2) hypervisor § Hosted hypervisor benefits: • Tackle So. C diversity • Reduced TTM • Zero virtualization overhead for personal phone. • Almost all components can be downloaded OTA (app store). 19 Copyright ® VMware, Inc. All Rights Reserved.

Device virtualization stack § Frontend (RHS) • Guest OS – VMM interface for virtual devices § Backend (LHS) • Hypervisor implementation of virtual device semantics on host 20

Frontend virtualization – where to slice? Application R: 137 G: 203 B: 223 Application level PV R: 192 G: 192 B: 192 Libraries, middleware Library level PV Operating System R: 248 G: 152 B: 29 Device level PV Device emulation Virtualization Layer 21 R: 217 G: 84 B: 30

Backend implementation § Host user space (application level) • Translate guest device I/Os to standard kernel/library calls (e. g. POSIX) • E. g. write ‘a’ to /dev/tty. S 0 for UART transmit ✓Portable, simple, language independent ✗System call overhead, latency § Host kernel • Translate guest device I/Os to kernel level function calls ✓Lowest overhead, direct access to device subsystem ✗Kernel dependent, modularity issues, increased TCB § Hardware (pass-thru) 22

Agenda § Smartphone hardware § I/O virtualization techniques § Examples from MVP hypervisor • Storage virtualization • GPS virtualization • Telephony virtualization 23

Storage virtualization 24

Smartphone storage devices § Internal storage • • NAND flash devices Software Flash Translation Layer (FTL) Limited size (256 MB – several GB) Kernel, application code, libraries, middleware § External storage • • • 25 micro. SD cards Hardware FTL Up to 32 GB today (2 TB future) Economics of semiconductor scaling Optimized for cost, media workloads Application data (and some code)

Storage virtualization stack 26

VM image storage on SD cards § Why? • Storage footprint of guest may be several GB (including checkpoint images) § Challenges • Performance • Reliability • Security 27

VM image storage on SD cards - performance § FTL optimized for cost, media workloads • Poor non-sequential I/O performance • VM workloads exhibit far more non-sequential I/O than MP 3 streaming, photos, videos, etc. 28

VM image storage on SD cards - performance. 29

VM image storage on SD cards - performance. 30

VM image storage on SD cards - performance. 31

VM image storage on SD cards – other challenges § Reliability • Dropped phone, whoops • Battery dies • Host software crashes (in particular at kernel level) § Security • SD cards are FAT formatted for compatibility • No access controls • Malicious host apps can read/write VM images 32

Log structured filesystems (LSF) § An excursion back in time to… 1991… • • • Slow seek latency on mechnical disks, CPUs getting faster Non-sequential file writes expensive Solution: every file update gets appended to a sequential “log” Maintain meta-data to later locate the most recent block for a file in the log Reads mostly hit in the page cache, don’t worry about non-sequentiality Mendel Rosenblum and John K. Ousterhout. The design and implementation of a log-structured file system. In ACM Symposium on Operating System Principles, 1991. § Benefits beyond performance… • Meta-data only updated after new data is on disk • Old data still exists earlier in log • Crash recovery discovers latest valid block on disk 33

LSF example 34

LSF example 35

LSF example 36

LSF example 37

Storage backend - Logging block store (LBS) § Backend represents VM disk & checkpoint images with files on host • • 38 Similar format to a LFS (performance, reliability) Split data and meta-data between internal & external storage Encryption & integrity checks on data file Meta-data and encryption keys located on protected internal storage (security)

LBS performance. 39

Agenda § Smartphone hardware § I/O virtualization techniques § Examples from MVP hypervisor • Storage virtualization • GPS virtualization • Telephony virtualization 40

GPS virtualization 41

GPS virtualization 42

Agenda § Smartphone hardware § I/O virtualization techniques § Examples from MVP hypervisor • Storage virtualization • GPS virtualization • Telephony virtualization 43

Telephony virtualization 44

Telephony virtualization § GSM • Dual SIM smartphones (geography dependent) • Multiple International Mobile Subscriber Identities (IMSIs) per SIM (geo dep) • GSM Alternate Line Service (ALS) (network dependent) § Network • Call forwarding, ala Google Voice, Line 2 § Voice-over-IP (Vo. IP) • Realistic with 4 G/LTE • Integration with enterprise Unified Commmunications (UC) 45