Kernel Structure and Infrastructure David Ferry Chris Gill

Kernel Structure and Infrastructure David Ferry, Chris Gill, Brian Kocoloski, Marion Sudvarg CSE 422 S - Operating Systems Organization Washington University in St. Louis, MO 63130

Kernel vs. Application Coding Two major differences: – The core kernel has no standard libraries – The core kernel must be a monolithic, statically linked library No standard libraries: – No libc (malloc, pthreads, string handling, etc. ) – Partly because of chicken/egg situation – Also, standard libraries can be too slow No dynamic loading of shared libraries for the core kernel CSE 422 S – Operating Systems Organization 2

Two Big Problems 1. Programmers rely on “library” functions for efficiency and correctness – E. g. , data structures, calls to sleep, etc. – What libraries to use? (hint: not glibc) • Kernel code has to be entirely self-contained • Also, chicken+egg problem, as glibc functions may in turn invoke the kernel via system calls 2. A totally static kernel would be enormous – About 20 million lines of code in 2015 – Most of this is hardware drivers that are never used on any given platform CSE 422 S – Operating Systems Organization 3

First Solution: Kernel “Libraries” Kernel libraries re-implement a lot of the functionality programmers expect in user space – Are statically compiled into the kernel – Automatically available just by including relevant header – Built to be kernel-safe (sleeping, waiting, locking, etc. is done properly) Features: – Utilities: kmalloc, kthreads, string parsing, etc. – Containers: hash tables, binary trees etc. – Algorithms: sorting, compression Mostly found under /lib and /include/linux CSE 422 S – Operating Systems Organization 4

Kernel “Libraries” Many kernel “libraries” have clear analogues to user space libraries: - malloc/free vs kmalloc/kfree - pthread_create vs kthread_create - sleep/usleep/nanosleep (user) vs msleep (kernel) Some, however, are a bit different: - e. g. , linked list implementation CSE 422 S – Operating Systems Organization 5

Example: Linked Lists (LKD pg. 88) struct fox { unsigned long tail_length; unsigned long weight; bool is_fantastic; }; If you wanted a list of structs of type struct_fox: struct fox { unsigned long tail_length; unsigned long weight; bool is_fantastic; struct fox * prev; struct fox * next; }; struct fox { unsigned long tail_length; unsigned long weight; bool is_fantastic; struct list_head list_node; }; CSE 422 S – Operating Systems Organization 6

Case 1: Manually Linked Together struct fox { unsigned long tail_length; unsigned long weight; bool is_fantastic; struct fox * prev; struct fox * next; }; Disadvantage: need to “roll your own code” for each list you create Ø Duplicate code throughout the kernel Ø Introduce bugs Ø Lose optimizations (placement within cache lines, etc. ) CSE 422 S – Operating Systems Organization 7

Case 2: Embedding Fox in list_head A list-able struct must contain struct list_head { struct list_head *next; struct list_head *prev; }; If you wanted a list of structs of type data: struct fox{ unsigned long tail_length; unsigned long weight; bool is_fantastic; struct list_head list_node; }; CSE 422 S – Operating Systems Organization 8

Source: https: //notes. shichao. io/lkd/ch 3/ CSE 422 S – Operating Systems Organization 9

Example: Linked Lists Initializing a list dynamically: struct fox *new_fox; new_fox = kmalloc(sizeof(struct fox), GFP_KERNEL); new_fox->tail_length = 40; new_fox->weight = 6; new_fox->is_fantastic = false; List head is initialized empty, i. e. points to itself INIT_LIST_HEAD(&(new_fox->list_node)); Or statically at compile time: static LIST_HEAD(fox_list_head); CSE 422 S – Operating Systems Organization 10

Example: Linked Lists Q: list_node. next, list_node. prev point to other list_head. So, how do I get access to the struct fox * embedded within? struct fox { unsigned long tail_length; unsigned long weight; bool is_fantastic; struct list_head list_node; }; list_node. prev struct list_head { struct list_head * prev; struct list_head * next; }; list_node. next struct list_head { struct list_head * prev; struct list_head * next; }; CSE 422 S – Operating Systems Organization 11

Example: Linked Lists Casting out from list_node to containing structure: struct fox * fox; struct list_head * node; fox = list_entry(node, struct fox, list_node); /* list_entry( arg 1: Pointer to list_head structure, arg 2: Type of outer “containing structure”, arg 3: Struct member of list_head within containing structure ) */ CSE 422 S – Operating Systems Organization 12

Example: Linked Lists Functions may take pointers to list nodes or pointers to the list head: Adding: struct data *new_data; list_add(&new_data->list, &data_list_head); Deleting: list_del(&new_data->list); kfree(new_data); //if dynamic CSE 422 S – Operating Systems Organization 13

Example: Linked Lists Iterating: struct list_head ptr; list_for_each(ptr, data_list_head){ //ptr points to each list structure } struct data *data_ptr; list_for_each_entry(d_ptr, data_list_head, list){ //d_ptr points to each data structure } Also: list_for_each_entry_reverse() list_for_each_entry_safe() //for modifying list elements CSE 422 S – Operating Systems Organization 14

Example: Linked Lists See also: – /include/linux/list. h – /include/linux/types. h Moral of the story: Always search for functionality before writing it yourself. CSE 422 S – Operating Systems Organization 15

Second Solution: Loadable Kernel Modules Kernel modules are kernel code that can be loaded dynamically: – Can be loaded/unloaded whenever – Runs in kernel mode – Can access exported kernel variables and functions – Can export variables and functions to kernel or other modules CSE 422 S – Operating Systems Organization 16

Why Kernel Modules Linux is a monolithic kernel. All functionality is compiled into the same static binary that is loaded into memory on boot. Without modules, the entire kernel would need to be loaded into memory to boot a node. Problems? Ø Waste of memory (embedded systems) Ø Slower boot time Ø Larger trusted computing base (TCB), more room for bugs CSE 422 S – Operating Systems Organization 17

What are Kernel Modules used for? • What pieces of code do we think might not be needed on every system that the kernel boots on? 1. Device Drivers 2. Architecture-specific code • Lines of code (just. c files) for: 1. Device Drivers: 1, 583, 159 2. Everything else: 1, 003, 777 • (Includes all of the architectures that we’re not using!) CSE 422 S – Operating Systems Organization 18

What are Kernel Modules used for? • Beyond space savings, what else might modules be useful for? 1. “Out-of-tree” functionality that is not accepted into “mainline” kernel 2. Allowing users to load custom functionality at runtime 3. Configuring/patching a running system without requiring a reboot 4. Your lab assignments in this class CSE 422 S – Operating Systems Organization 19

Module Implementation Must define: – An initialization function called on load – An exit function called on unload The init function must be self contained! – Must unwind actions if initialization cannot complete successfully – E. g. if you kmalloc() space but don’t free it, that physical memory is now lost (until system restart) You can also pass parameters to modules at load time. CSE 422 S – Operating Systems Organization 20

Loading and Unloading Modules Today we will use insmod and rmmod – Must use sudo to invoke these on your Rpi ERROR: could not insert module jiffies_module. ko: Operation not permitted – Cannot unload a module that’s not loaded ERROR: Module jiffies_module is not currently loaded – Cannot load a module that’s already there ERROR: could not insert module simple_module. ko: File exists CSE 422 S – Operating Systems Organization 21