Chapter 1 Kernel Overview Objectives Review development and

Chapter 1: Kernel Overview

Objectives • Review development and history of UNIX and Linux. • Clarify the nature and extent of the kernel within the context of the operating system proper. • Identify operating system design goals and tradeoffs. • Review established architectural approaches for implementing an operating system. • Introduce the notion of the “core” Linux kernel. • Understand the Linux approach to hardware (architecture) abstraction and independence. • Explore the layout of the Linux source code tree. 2

History • • • UNIX: 1969 Thompson & Ritchie AT&T Bell Labs BSD: 1978 Berkeley Software Distribution Commercial Vendors: Sun, HP, IBM, SGI, DEC GNU: 1984 Richard Stallman, FSF POSIX: 1986 IEEE Portable Operating System un. IX Minix: 1987 Andy Tannenbaum SVR 4: 1989 AT&T and Sun Linux: 1991 Linus Torvalds Intel 386 (i 386) Open Source: GPL, LGPL, Cathedral and the Bazaar 3

Linux Features • “UNIX-like” operating system • “aims at” standards compliance • “all the features you would expect in a modern UNIX” – – – preemptive multitasking virtual memory (protected memory, paging) shared libraries demand loading, dynamic kernel modules shared copy-on-write executables TCP/IP networking • other features: – – – SMP support, large memory, large files advanced networking, advanced filesystems efficient, stable, highly portable, supports most device hardware active development community, support, documentation, open source GUIs, applications 4

What’s a Kernel? • aka: executive, system monitor, nucleus • controls and mediates access to hardware • implements and supports fundamental abstractions • processes, files, devices, users, net, etc. • schedules “fair” sharing of system resources • memory, cpu, disk, descriptors, etc. • enforces security and protection • responds to user requests for service (system calls) • performs routine maintenance, system checks, etc. 5

Kernel Design Goals • performance: efficiency, speed – utilize resources to capacity, low overhead, code size • stability: robustness, resilience – uptime, graceful degradation • capability: features, flexibility, compatibility • security, protection – protect users from each other, secure system from bad guys • portability • clarity • extensibility 6

Design Tradeoffs • Butler Lampson: “choose any three design goals” • efficiency vs. protection – more checks, more overhead • clarity vs. compatibility – ugly implementation of “broken” standards (e. g. signals) • flexibility vs. security – the more you can do, the more potential security holes! • not all are antagonistic – portability tends to enhance code clarity 7

Waterloo Diagrams mm ipc fs sched vfs sched ipc net mm net Conceptual Concrete 8

Stephen Tweedie’s Diagram User Processes Scheduler Process Manager Socket Manager Syscalls VM VFS Network Protocols File Systems Packet Requestor IO Requestor Net Devices Memory Allocator Traps Math Support Char Devices Block Devices 9

Vahalia’s Diagram from Unix Internals: The New Frontiers Uresh Vahalia / Prentice-Hall 1996 a. out elf coff file NFS device anonymous virtual memory framework exec switch FFS vnode/vfs interface s 5 fs core utilities block device switch disk scheduler framework STREAMS real-time tape network time-sharing tty system 10

“Core” Kernel Applications System Libraries (libc) Modules System Call Interface I/O Related File Systems Networking Device Drivers Process Related Scheduler Memory Management IPC Architecture-Dependent Code Hardware 11

Architectural Approaches • • • monolithic layered modularized micro-kernel virtual machine 12

Isolating Hardware Dependencies • architecture (cpu) – dependent (/arch) – independent (everything else) • • abstract dependencies behind functions and macros link appropriate version at compile-time device-dependencies isolated in device drivers provide general abstractions that map to reality – e. g. three-level page tables • tradeoff: exploiting special hardware features 13

Source Tree Layout Documentation scripts ipc net init arch drivers alpha arm i 386 ia 64 m 68 k mips 64 ppc s 390 sh sparc 64 /usr/src/linux acorn atm block cdrom char dio fc 4 i 2 c i 2 o ide ieee 1394 isdn macintosh misc net … fs adfs affs autofs 4 bfs code cramfs devpts efs ext 2 fat hfs hpfs … kernel lib mm include asm-alpha asm-arm asm-generic asm-i 386 asm-ia 64 asm-m 68 k asm-mips 64 … linux math-emu net pcmcia scsi video adfs affs autofs 4 bfs code cramfs devpts efs ext 2 fat hfs hpfs … 802 appletalk atm ax 25 bridge core decnet econet ethernet ipv 4 ipv 6 ipx irda khttpd lapb … 14

linux/Documentation • spotty but important collection of developer-generated documentation; you need to read what’s in here! • recent effort to produce javadoc-style documentation from source header comments using Open. Doc • an ambitious open-source kernel book effort has begun; see kernelbook. sourceforge. net for details • some especially interesting entries: – – – kernel-docs. txt (a bit out of date but good) filesystems/ (very extensive) networking/ (very extensive) kmod. txt oops-tracing. txt spinlocks. txt (the official story from Linus) 15

linux/arch • subdirectories for each current port • each contains kernel, lib, mm, boot and other directories whose contents override code stubs in architecture independent code • lib contains highly-optimized common utility routines such as memcpy, checksums, etc. • arch as of 2. 4: – alpha, arm, i 386, ia 64, m 68 k, mips 64 – ppc, s 390, sh, sparc 64 16

linux/drivers • • • largest amount of code in the kernel tree (~1. 5 M) device, bus, platform and general directories drivers/char – n_tty. c is the default line discipline drivers/block – elevator. c, genhd. c, linear. c, ll_rw_blk. c, raid. N. c drivers/net –specific drivers and general routines Space. c and net_init. c • drivers/scsi – scsi_*. c files are generic; sd. c (disk), sr. c (CDROM), st. c (tape), sg. c (generic) • general: – cdrom, ide, isdn, parport, pcmcia, – pnp, sound, telephony, video • buses – fc 4, i 2 c, nubus, pci, sbus, tc, usb • platforms – acorn, macintosh, s 390, sgi 17

linux/fs • contains: – virtual filesystem (VFS) framework – subdirectories for actual filesystems • vfs-related files: – exec. c, binfmt_*. c - files for mapping new process images – devices. c, blk_dev. c – device registration, block device support – super. c, filesystems. c – inode. c, dcache. c, namei. c, buffer. c, file_table. c – open. c, read_write. c, select. c, pipe. c, fifo. c – fcntl. c, ioctl. c, locks. c, dquot. c, stat. c 18

linux/include • include/asm-* – architecture-dependent include subdirectories • include/linux – header info needed both by the kernel and user apps – usually linked to /usr/include/linux – kernel-only portions guarded by #ifdefs • #ifdef __KERNEL__ • /* kernel stuff */ • #endif • other directories: – – – math-emu net pcmcia scsi video 19

linux/init • just two files: version. c, main. c • version. c – contains the version banner that prints at boot • main. c – architecture-independent boot code • start_kernel is the primary entry point 20

linux/ipc • System V IPC facilities • if disabled at compile-time, util. c exports stubs that simply return –ENOSYS • one file for each facility: – sem. c – semaphores – shm. c – shared memory – msg. c – message queues 21

linux/kernel • the core kernel code • sched. c – “the main kernel file” – scheduler, wait queues, timers, alarms, task queues • process control – fork. c, exec. c, signal. c, exit. c – acct. c, capability. c, exec_domain. c • kernel module support – kmod. c, ksyms. c, module. c • other operations – time. c, resource. c, dma. c, softirq. c, itimer. c – printk. c, info. c, panic. c, sysctl. c, sys. c 22

linux/lib • kernel code cannot call standard C library routines • files: – – – brlock. c – “Big Reader” spinlocks cmdline. c – kernel command line parsing routines errno. c – global definition of errno inflate. c – “gunzip” part of gzip. c used during boot string. c – portable string code • usually replaced by optimized, architecture-dependent routines – vsprintf. c – libc replacement 23

linux/mm • paging and swapping – swap. c, swapfile. c (paging devices), swap_state. c (cache) – vmscan. c – paging policies, kwapd – page_io. c – low-level page transfer • allocation and deallocation – slab. c – slab allocator – page_alloc. c – page-based allocator zone allocator – vmalloc. c – kernel virtual-memory allocator • memory mapping – memory. c – paging, fault-handling, page table code – filemap. c – file mapping – mmap. c, mremap. c, mlock. c, mprotect. c 24

linux/net • changing too fast! • i haven’t figured it out yet 25

linux/scripts • scripts for: – menu-based kernel configuration – kernel patching – generating kernel documentation 26

Sizes (linux-2. 4. 0 -test 2) size directory entries files loc 90 M 4. 5 M 16. 5 M 54 M 5. 6 M 14. 2 M 28 K 120 K 332 K 80 K 356 K 5. 8 M 400 K /usr/src/linux/ Documentation arch drivers fs include init ipc kernel lib mm net scripts 19 97 12 31 70 19 2 6 25 8 19 33 26 7645 380 1685 2256 489 2262 2 6 25 8 19 453 42 2. 6 M na 466 K 1. 5 M 150 K 285 K 1 K 4. 5 K 12 K 2 K 162 K 12 K 27

Summary • Linux is a modular, UNIX-like monolithic kernel • Kernel is the heart of the OS that executes with special hardware permission (kernel mode) • “Core kernel” provides framework, data structures, support for drivers, modules, subsystems • Kernel designers must consider many competing goals • Linux source tree mirrors kernel structure • Architecture dependent source subtrees live in /arch • “main” lives in /kernel/init. c • lxr. linux. no is a nice web-based source browser 28