Porting Palacios to the Linux Kernel EECS 441

Porting Palacios to the Linux Kernel EECS 441 – Resource Virtualization Steven Chen, Jason Lee, Pill Park

Overview Goals and Thoughts Visualization Approach Status Implementation Functions Difficulties Looking to the Future Acknowledgements, Thanks Questions Chen, Lee, Park - Team Linux, EECS 441

Goals and Thoughts Have the Palacios VMM embedded into Linux Have Palacios compiled into Linux Have a guest successfully run in Palacios on Linux kernel host for Palacios Would facilitate more widespread usage of Palacios Additional platform for Palacios Current host OSes with Palacios embedded Kitten LWK (Sandia National Labs) Geek. OS (University of Maryland) Chen, Lee, Park - Team Linux, EECS 441

Visualization Palacios on Kitten Palacios on Linux Applications Guest OS Palacios VMM Kitten LWK Linux Kernel x 86 -64, Cray XT IA-32, x 86 -64 Chen, Lee, Park - Team Linux, EECS 441

Approach Initial Approach Pattern matching between Kitten LWK, Linux kernel Tedious process May eventually be necessary anyways Revised Approach Statically link Palacios to Linux build process Troubleshoot boot process Troubleshoot Palacios load process Load guest blob Deal with panics Deal with unimplemented functions (pattern matching? ) Get to a terminal and working guest eventually Chen, Lee, Park - Team Linux, EECS 441

Status Statically link Palacios to Linux build process Done Troubleshoot boot process Troubleshoot Palacios load process Load guest blob Deal with panics In Progress Deal with unimplemented functions (pattern matching? ) Done In Progress Get to a terminal and working guest eventually In Progress Chen, Lee, Park - Team Linux, EECS 441

Status – Serial Output Chen, Lee, Park - Team Linux, EECS 441

Implementation Ø Statically Ø Linking Added lines to Makefile Ø Links libv 3 vee. a into the Linux kernel 666: # Link the LWK with the Palacios virtual machine monitor 667: libs-$(CONFIG_PALACIOS) += --wholearchive $(shell echo $(CONFIG_PALACIOS_PATH)/libv 3 vee. a ) --no-whole-archive Starting Palacios Added lines to init/main. c 847: int palacios_init(void); 896: #ifdef CONFIG_PALACIOS 897: palacios_init(); 898: #endif Chen, Lee, Park - Team Linux, EECS 441

Functions – Adding print functionality arch/x 86/kernel/palacios. c 20 static void 21 palacios_print( 22 const char * fmt, 23 . . . 24 ) 25 { 26 va_list ap; 27 va_start(ap, fmt); 28 vprintk(fmt, ap); 29 va_end(ap); 30 31 return; 32 } Chen, Lee, Park - Team Linux, EECS 441

Functions – Panic for diagnostics arch/x 86/kernel/palacios. c 210 static void 211 palacios_interrupt_cpu( 212 struct guest_info * vm, 213 int cpu_id 214 ) 215 { 216 panic("palacios_interrupt_cpu"); 217 return; 218 } Chen, Lee, Park - Team Linux, EECS 441

Functions – Migrating Palacios’ Code arch/x 86/kernel/palacios. c alloc/free for Kitten alloc/free for Linux static void * palacios_alloc(unsigned int size) { { return kmem_alloc(size); return kmalloc(size, GFP_KERNEL); } } static void palacios_free(void * addr) { { return kmem_free(addr); } kfree(addr); return; } Chen, Lee, Park - Team Linux, EECS 441

Functions – Allocating Pages arch/x 86/kernel/palacios. c static void * palacios_allocate_pages(int num_pages) { int order = 0; bool extra = false; printk("will get %d pagesn", num_pages); int i = 0; for(; i < 20; i++){ int temp = num_pages; if(num_pages > 1) { num_pages = num_pages / 2; order++; } if(num_pages == 1) { if(extra) {order++; } break; } if((temp % 2) == 1) { extra = true; } } Chen, Lee, Park - Team Linux, EECS 441 if(order >= 14){ printk("Asked for order==%d, getting order==%d pagesn", order, 14); order = 14; } void *p = __get_free_pages(GFP_KERNEL , order); if(p){ head = insert(head, (long) p, or der); p = __pa(p); return p; }

Functions – Freeing Pages arch/x 86/kernel/palacios. h Will use a binary search tree to recall pages to be freed struct node { struct node* left; struct node* right; long address; int valid; int order; }; extern struct node* lookup(struct node* node, long start_addr); extern struct node* New. Node(long ptr, int order); extern struct node* insert(struct node* node, long ptr, int order); Chen, Lee, Park - Team Linux, EECS 441

Functions – Freeing Pages arch/x 86/kernel/palacios. c static void palacios_free_page( void * page_paddr ) { struct node* node = lookup(head, page_paddr); free_pages(page_paddr, node->order); node->valid = 0; } Chen, Lee, Park - Team Linux, EECS 441

Functions – Freeing Pages arch/x 86/kernel/palacios. c struct node* lookup(struct node* node, long start_addr) { if (!node) { return 0; } else { if (start_addr == node->address) { return node; } else { if (start_addr < node->address) { return lookup(node->left, start_addr); } else { return lookup(node->right, start_addr); } } }//end of the function Chen, Lee, Park - Team Linux, EECS 441

Functions – Making it by Faking it arch/x 86/kernel/palacios. c Getting to larger issues by hacking around smaller issues… static unsigned int palacios_get_cpu_khz(void) { printk("palacios_get_cpu_khz() lying to Palacios and saying 1 GHzn"); return 1000000; } Chen, Lee, Park - Team Linux, EECS 441

Difficulties OS Hooks to Interrupts /** Structure used by the Palacios hypervisor to interface with the host kernel. */ struct v 3_os_hooks palacios_os_hooks = { . print = palacios_print, . allocate_pages = palacios_allocate_pages, . free_page = palacios_free_page, . malloc = palacios_alloc, . free = palacios_free, . vaddr_to_paddr = palacios_vaddr_to_paddr, . paddr_to_vaddr = palacios_paddr_to_vaddr, . hook_interrupt = palacios_hook_interrupt, . ack_irq = palacios_ack_interrupt, . get_cpu_khz = palacios_get_cpu_khz, . start_kernel_thread = palacios_start_kernel_thread, . yield_cpu = palacios_yield_cpu, . mutex_alloc = palacios_mutex_alloc, . mutex_free = palacios_mutex_free, . mutex_lock = palacios_mutex_lock, . mutex_unlock = palacios_mutex_unlock, . get_cpu = palacios_get_cpu, . interrupt_cpu = palacios_interrupt_cpu, . call_on_cpu = palacios_xcall, . start_thread_on_cpu = palacios_start_thread_on_cpu, }; Chen, Lee, Park - Team Linux, EECS 441

Difficulties – Keyboard IRQs arch/x 86/kernel/palacios. c // hook keyboard host events for deliver to palacios int i = 1; for(; i < 200; i++){ error = request_irq( i, &palacios_keyboard_interrupt, 0, "keyboard", NULL ); if(!error){ break; } }//for if(error){ printk("request irq for keyboard failedn"); panic("request keyboard irq failed"); } Chen, Lee, Park - Team Linux, EECS 441

Looking to the Future Currently, 64 MB limit on page_alloc 214 [16 K] * 4 KB pages = 64 MB Short term - make a small guest Long term – resolve memory limits Properly Implement Interrupts Improve the Palacios-Linux interface Look to the Palacios-Kitten interface for reference Have our code utilize Palacios data structures Support multi-core execution Chen, Lee, Park - Team Linux, EECS 441

Acknowledgements, Thanks Professor Peter Dinda UA Andy Gocke Lei Xia V 3 VEE development group Chen, Lee, Park - Team Linux, EECS 441

Questions? Chen, Lee, Park - Team Linux, EECS 441