Linux UNIX System V Release 4 SVR 4

Ιστορική αναδρομή • Το Linux περιλαμβάνει χαρακτηριστικά από πολλά γνωστά λειτουργικά συστήματα UNIX όπως: • System V Release 4 (SVR 4) - AT&T • 4. 4 BSD release - University of California at Berkeley (4. 4 BSD), • Digital Unix - DEC Compaq HP, • AIX - IBM, • HP-UX - Hewlett-Packard, • Solaris - Sun Microsystems 2

Διανομές Linux • Standard, precompiled sets of packages, or distributions, include the basic Linux system, system installation and management utilities, and ready-toinstall packages of common UNIX tools. • The first distributions managed these packages by simply providing a means of unpacking all the files into the appropriate places; modern distributions include advanced package management. • Red Hat, Debian, Su. SE, Mandrake are popular distributions from commercial and noncommercial sources, respectively (see www. linux. org). • The RPM Package file format permits compatibility among the various Linux distributions (see www. linuxbase. org). 6

Which distribution to use ? • Red. Hat (www. redhat. com) – Big, professional, very widely used • Debian (www. debian. org/) – Open development model, excellent packaging system • Mandrake (www. mandrakesoft. com) – Aims to be very easy to install and use • Su. SE (www. suse. com/) – Compromise between Red Hat and Mandrake • Slackware (www. slackware. com/) – Most traditional; little extra help 7

Αρχιτεκτονική πυρήνα Linux User Programs Trap User level Libraries Kernel level System Call Interface File System Management IPC Buffer Cache Process Mgt. Scheduling Memory Mgt. Device Drivers Hardware Control (Interrupts handling, etc) Hardware 29

Δομή του πυρήνα User application Standard Unix libraries User level Kernel level System call interface File Subsystem Buffer cache Process control system Interprocess communication scheduler Memory management Character block Device drivers Hardware control Hardware level Hardware 30

Scheduling 35

• The job of allocating CPU time to different tasks within an operating system. • While scheduling is normally thought of as the running and interrupting of processes, in Linux, scheduling also includes the running of the various kernel tasks. • Running kernel tasks encompasses both tasks that are requested by a running process and tasks that execute internally on behalf of a device driver. 36

Kernel Synchronization • A request for kernel-mode execution can occur in two ways: – A running program may request an operating system service, either explicitly via a system call, or implicitly, for example, when a page fault occurs. – A device driver may deliver a hardware interrupt that causes the CPU to start executing a kerneldefined handler for that interrupt. • Kernel synchronization requires a framework that will allow the kernel’s critical sections to run without interruption by another critical section. 37

Kernel Synchronization (Cont. ) • Linux uses two techniques to protect critical sections: 1. Normal kernel code is nonpreemptible – when a time interrupt is received while a process is executing a kernel system service routine, the kernel’s need_resched flag is set so that the scheduler will run once the system call has completed and control is about to be returned to user mode. 2. The second technique applies to critical sections that occur in an interrupt service routines. – By using the processor’s interrupt control hardware to disable interrupts during a critical section, the kernel guarantees that it can proceed without the risk of concurrent access of shared data structures. 38

Kernel Synchronization (Cont. ) • To avoid performance penalties, Linux’s kernel uses a synchronization architecture that allows long critical sections to run without having interrupts disabled for the critical section’s entire duration. • Interrupt service routines are separated into a top half and a bottom half. – The top half is a normal interrupt service routine, and runs with recursive interrupts disabled. – The bottom half is run, with all interrupts enabled, by a miniature scheduler that ensures that bottom halves never interrupt themselves. – This architecture is completed by a mechanism for disabling selected bottom halves while executing normal, foreground kernel code. 39

Interrupt Protection Levels • Each level may be interrupted by code running at a higher level, but will never be interrupted by code running at the same or a lower level. • User processes can always be preempted by another process when a time-sharing scheduling interrupt occurs. 40

Process Scheduling • Linux uses two process-scheduling algorithms: – A time-sharing algorithm for fair preemptive scheduling between multiple processes – A real-time algorithm for tasks where absolute priorities are more important than fairness • A process’s scheduling class defines which algorithm to apply. • For time-sharing processes, Linux uses a prioritized, credit based algorithm. – The crediting rule factors in both the process’s history and its priority. – This crediting system automatically prioritizes interactive or I/Obound processes. 41

Process Scheduling (Cont. ) • Linux implements the FIFO and round-robin real-time scheduling classes; in both cases, each process has a priority in addition to its scheduling class. – The scheduler runs the process with the highest priority; for equal-priority processes, it runs the process waiting the longest – FIFO processes continue to run until they either exit or block – A round-robin process will be preempted after a while and moved to the end of the scheduling queue, so that round-robing processes of equal priority automatically time-share between themselves. 42

File Systems 43

VFS Sun Microsystems introduced the virtual file system framework in 1985 to accommodate the Network File System cleanly. – VFS allows diverse specific file systems to coexist in a file tree, isolating all FS-dependencies in pluggable filesystem modules. user space syscall layer (file, uio, etc. ) netwo rk proto col stack (TCP/ IP) Virtual File System (VFS) NFS FFS LFS *FS ext 2. xfs device drivers VFS was an internal kernel restructuring with no effect on the syscall interface. Incorporates object-oriented concepts: a generic procedural interface with multiple implementations. Based on abstract objects with dynamic method binding by type. . . in C. Other abstract interfaces in the kernel: device drivers, file objects, executable files, memory objects. 45

Linux VFS context 46

How it looks logically… User space Inode Cache VFS Directory Cache NTFS Ext 2 Buffer Cache Crypt. FS Maintained by the kernel Disk Drivers 47

Task 2 Task 1 … Task n user space kernel space VIRTUAL FILE SYSTEM minix ext 2 msdos proc Buffer Cache device driver for hard disk device driver for floppy disk Linux Kernel software Hard Disk Floppy Disk hardware 48

File System • Linux files are organized by a hierarchy of labels, commonly known as a directory structure. The files referenced by these labels may be of three kinds: – Regular files, which contains a sequence of bytes that generally corresponds to code (programs) or data. – Directory files, which are stored on disk in a special format and form the backbone of the file system – Special file, which correspond to peripherals such as printers or disks. • To the user, Linux file system appears as a hierarchical directory tree obeying UNIX semantics. 49

File System • / is the root directory; reference point for all directories. Every file has a unambiguous pathname: – /home/user 1/papers 50

Directories σε συστήματα UNIX • • /bin Binaries which are absolutely essential to run Linux. /boot All the files required for booting Linux on a system. /dev All the devices have their corresponding files here. /etc All the configuration files for the various software stored here. Don't play with this directory. • /home All users will have their 'My Documents' under this directory. If your id is studin, your 'My Documents' (called homedirectory) is /home/studin. • /lib The libraries required by system-applications. (Just like DLLs in Windows. ) • /lost+found When a disk-check finds files which are damaged or which are not linked to any directory, they are recovered to this directory. Such damages are almost always due to incorrect shutdown. 51

Directories σε συστήματα UNIX • /mnt The directory where peripherals and other file-systems are mounted. • /opt The directory where optional software installed. • /proc houses a pseudo-file system. Its contents really do not exist anywhere on the disk, and are made available only when you cd to this directory and look at some file. • /root The home-directory for the super-user: root. • /sbin The system-administration binaries exist here. • /tmp The directory where temporary files are created and stored. All users can save files here. • /usr Everything related to users • /var Files whose contents vary frequently are in this directory. 52

• /usr Everything related to users – /usr/bin houses critical binaries of the users – /usr/include The header-files required by programs for compilation. – /usr/lib The libraries required by user-applications. – /usr/local Files peculiar to this particular machine. – /usr/sbin User-administration binaries. – /usr/share Information that can be shared by most users. – /usr/src The source-code for the Linux kernel. – /usr/X 11 R 6 Files needed by the X Window system. • /var Files whose contents vary frequently are in this directory. – /var/log The log-files of the system. – /var/spool Directories for mail, news, printing and other queued work. 53

Hard Disk Partitioning • • Partitioning is a means of sub-dividing a hard disk. A hard disk must contain at least one partition. There are two types of partitions – primary and extended. A disk can contain up to 4 primary partitions or up to 3 primary and 1 extended partition. • One primary partition may be designated as active, this will be used for booting the computer. • Different operating systems can be installed on different primary partitions – allowing multi-booting. • An extended partition cannot be accessed directly by the operating system but is instead sub-divided into logical drives. 54

Disk Partitioning : fdisk • Partition is large group of sectors allocated for a specific purpose – IDE disks limited to 4 physical partitions – Logical (extended) partition inside physical partition • Specify number of cylinders to use • Specify type / (root) – Magic number recognized by OS usr home tmp 55

Hard Disk Initialization • In order to be usable by the OS a partition must be initialized. • In UNIX/Linux the mkfs command is used, in Windows the command is FORMAT. • The initialisation process sets up the necessary structures on a disk partition. • In UNIX/Linux an initialized partition is referred to as a file system on Windows it is called a drive. 56

File System Creation (Format) • Low-level Format • Logical Format: Create file system structure – Super block • Magic number, block size and etc. • Bitmaps (blocks, i-node), • Pointers to i-node blocks (or FAT) – (i-nodes) • Tools – Windows: format – Linux: mke 2 fs 57

UNIX/Linux File System – Volume Structure • When the mkfs command is run to initialize a partition the following layout is created: – boot block: first stage boot program. – super block: contains details of disk size and allocation of inodes and data blocks. – i-nodes: storage space for inodes. – data Blocks: rest of disk available for directories and files. 58

UNIX/Linux File System – Free Space Control • When the mkfs command is run it creates a chain of linked blocks. • The super block contains the start block in the chain. • Each block contains an array of free data block numbers and a pointer to the next block in the chain. 59

Mounting • Separate file systems • After mounting Before mounting After mounting 60

Logical Volumes • File systems on physical volumes (partitions) – A sequential series of blocks on a physical disk. – Historically, a partition size is static. • Shortcomings? – What should be the size of a partition? – Can a partition be resized when needed? – Can a file system be built over multiple disks? • References – http: //www. suse. de/en/whitepapers/lvm/ – http: //ds 9 a. nl/lvm-howto/ – http: //www. tldp. org/HOWTO/LVM-HOWTO/ 61

Logical Volume Management • A layer of abstraction over the storage: – LVM re-organizes disk blocks in physical partitions into logical volumes. /dev/hda 1, /dev/sda 2 … /dev/vg 0/usrlv, /dev/vg 0/varlv, /dev/vg 0/homelv – A file system is installed on a logical volume • Separates hardware and software storage management – LVM hides the hardware details – May change configurations during runtime. 62

Filesystem Comparison Maximal FS size Maximal filesize Minix Ext Xia Ext 2 64 MB 2 GB 4 TB 64 MB 2 GB 64 MB Maximal filename 14/30 chars 255 chars 248 chars 2 GB 255 chars 3 timestamps no no yes Extensible? no no no yes Can vary block size? no no no yes Code is maintained? yes no ? yes 63

Common concepts • • Files are represented by inodes Directories are special files (dentries) Devices accessed by I/O on special files UNIX filesystems can implement ‘links’ 64

i-nodes • A structure that contains file’s description: – – – Type Access rights Owners Timestamps Size Pointers to data blocks • Kernel keeps the inode in memory (open) 65

i-node diagram inode Direct blocks File info Indirect blocks Double Indirect Blocks 66

Directories • • These are structured in a tree hierarchy Each can contain both files and directories A directory is just a special type of file Special user-functions for directory access Each dentry contains filename + inode-no Kernel searches the direrctory tree translates a pathname to an inode-number 67

Directory diagram i-node Table Directory i 1 name 1 i 2 name 2 i 3 name 3 i 4 name 4 68

Links • Multiple names can point to same inode • The inode keeps track of how many links • If a file gets deleted, the inode’s link-count gets decremented by the kernel • File is deallocated if link-count reaches 0 • This type of linkage is called a ‘hard’ link • Hard links may exist only within a single FS • Hard links cannot point to directories (cycles) 69

Symbolic Links • • • Another type of file linkage (‘soft’ links) Special file, consisting of just a filename Kernel uses name-substitution in search Soft links allow cross-filesystem linkage But they do consume more disk storage 70

Filesystem performance • Two predominant performance criteria: – Speed of access to file’s contents – Efficiency of disk storage utilization • How can these be meaningfully measured • How do we screen out extraneous factors – The underlying hardware medium – The user-interface software, etc 71