ITEC 400 Week Eight Professor Robert DAndrea Week

ITEC 400 Week Eight Professor Robert D’Andrea

Week Nine Agenda Administrative Issues This Week’s Expected Outcomes Review Pipes and Parallelism Review for final exam Moving around in UNIX Break-Out Problems Upcoming Deadlines Hands-on Information Lab Assistance, Questions, and Answers

This Week’s Expected Outcome Upon successful completion of this module, the student will be more knowledgeable about the: File system integrity /proc directories ASCII and EBCDIC Character set CMOS BIOS Linux boot-up procedure (System 5) Red Hat Enterprise Release 6. 0 boot-up procedure (Upstart)

Unix Operating System http: //en. wikipedia. org/wiki/Firmware This Web site contains information on: - The evolution of firmware Uses of firmware: - Cell phones (Nexus Smart phone) - Drones - Map navigator - Laptops Firmware hacking is also another avenue used to gain control of systems.

Unix Operating System http: //www. driver-fix. com/driverdetective. php? t=Firmware Diver. Detective Download driver updates automatically and safely. This software utility will keep your system up-to-date by automatically retrieving the latest drivers from your hardware’s manufacturer.

Unix Operating System Firmware is a computer program that is embedded in the hardware device. Firmware can be stored in flash memory or as a binary image file that can be uploaded onto existing hardware by a user. In summary, firmware is a combination of hardware and software.

Unix Operating System Firmware applications include the following: - BIOS found in IBM-compatible PCs. - Open Firmware, used in computers marketed by Sun Microsystems and Apple Computers. - Computer peripherals - The i. Pod’s control menus. - Automobiles with on-board computers/sensors - Cisco IOS

UNIX Operating System Let’s use the pipe commands below as our model. Command A | Command B | Command-C The Linux system is designed with a separate file system named, pipefs. The pipefs file system is mounted alongside the root file system in kernel space. Applications should be designed to consume data as soon as it becomes available (written) and the writing process isn’t interrupted or blocked. The pipe size is an important factor and should be known. Most Linux system default settings are 65, 536 bytes.

UNIX Operating System Next, let’s focus on performance. If a process A is writing data at roughly the same speed as process B is reading it and process C is handling it (this is parallel processing), pipes work very well and are considered running at high performance. The last thing you want to happen is an imbalance between the three processes. This is referred to as a pipeline. A pipeline works well with balanced buffering and when data is streaming. If a process like a sort command is used, the whole pipeline is interrupted until the sort command finishes sorting all data.

UNIX Operating System Unix File Descriptors Unix has three file descriptors 0 – Input (stdin) 1 – Output (stdout) 2 – Error (stderr) cat /~dandrear/Winter_2017_Solutions/text_file. txt Hello Franklin University 2017

UNIX Operating System In the previous example, the cat command displayed the contents of the text_file. txt file. The output of the cat command was displayed or re-directed to the terminal which is the output file descriptor by default. To prove it, lets try this:

UNIX Operating System Enter: tty /dev/pts/1 Enter: cat /~dandrear/Fall_2017_Solutions/text_file. txt > /dev/pts/1 Hello Franklin University 2017

UNIX Operating System The tty command provides the terminal information. As shown previously, when we specify the /dev/pts/1 as the output file, the contents are displayed on the terminal. The output of a command can be re-directed to a file or a terminal using the re-direction operator (>). Also, note the value of the output file descriptor "1" is not specified previously. It is so because the symbol ">" defaults to output.

UNIX Operating System cat /~dandrear/Fall_2017_Solutions/text_file. txt 1> /dev/pts/1 Hello Franklin University 2017 In this case, the output file descriptor is specified explicitly to redirect the output to a terminal. Note: There should be no space between 1 and > symbol. If a space is provided, the cat command treats 1 as a filename.

UNIX Operating System Similarly, to re-direct the output of a command to another file: cat /~dandrear/Fall_2017_Solutions/text_file. txt 1> /tmp/text_file 2. txt cat /tmp/text_file 2. txt Hello Franklin University 2017

UNIX Operating System cat /~dandrear/Fall_2017_Solutions/text_file. txt > /tmp/text_file 2. txt /tmp>cat /tmp/text_file 2. txt Hello Franklin University 2017

UNIX Operating System Error File Descriptor Let us try to display the content of a non-existent file. /tmp>cat abc. txt cat: abc. txt: No such file or directory The above shown output is an error message of the cat command. Please note that the error message also got re-directed to the terminal which means the terminal is the default file descriptor for the errors.

UNIX Operating System Let us try to re-direct this error message to a file as in the case of output redirection: >cat abc. txt > /tmp/abc_error. txt cat: abc. txt: No such file or directory >cat /tmp/abc_error. txt /export/home/dandrear/Fall_2017_Solutions >

UNIX Operating System What happened? The error message did not get re-directed to the file /tmp/abc_error. txt. As we said earlier, when we use ">", only the output gets re-directed to the terminal. /tmp>cat abc. txt 2> /dev/null In the above example, no error message is displayed on the terminal because the errors are now redirected to a null terminal.

UNIX Operating System One of the common instances where we don’t intend to use error messages is during the find command. When find searches in directories where the user does not have the read permissions, it outputs an error message. To redirect those errors: find. -mtime -10 2>/dev/null

UNIX Operating System Input File Descriptor Now, lets come to our last part: input descriptor. Like output and error, the terminal is the default input descriptor. /tmp>read x 20 /tmp>echo $x 20

UNIX Operating System The read command expects input from the user. When the read command is typed in, it prompts the user for an input. The input value is displayed on the terminal, which in turn is saved in the variable x in this case. The input can also come from a file.

UNIX Operating System /tmp>cat /~dandrear/Fall_2017_Solutions/text_file 20. txt 20 /tmp>read x < /~dandrear/Fall_2017_Solutions/text_file 20. txt /tmp>echo $x 20 The file text_file. txt contains the value 20. The read command is made to take the input value from file text_file. txt. Note the symbol "<" is used for input re-direction. The value in turn is saved in the variable x.

UNIX Operating System If you run fsck, the filesystem check and repair command, it might find data fragments that are not referenced anywhere in the filesystem. In particular, fsck might find data that looks like a complete file but doesn't have a name on the system — an inode with no corresponding file name. This data is still using up space, but it isn't accessible by any normal means.

UNIX Operating System If you tell fsck to repair the filesystem, it will turn these almost-deleted files back into files. The thing is, the file had a name and location once, but that information is no longer available. So fsck deposits the file in a specific directory, called lost+found.

UNIX Operating System Files that appear in lost+found are typically files that were already unlinked (i. e. their name had been erased) but still opened by some process (so the data wasn't erased yet) when the system halted suddenly (kernel panic or power failure). If that's all that happened, these files were slated for deletion anyway, you don't need to care about them.

UNIX Operating System Files can also appear in lost+found because the filesystem was in an inconsistent state due to a software or hardware bug. If that's the case, it's a way for you to find files that were lost but that the system repair managed to salvage. The files may or may not contain useful data, and even if they do they may be incomplete or out of date; it all depends how bad the filesystem damage was.

UNIX Operating System On many filesystems, the lost+found directory is a bit special because it pre-allocates a bit of space for fsck to deposit files there. (The space isn't for the file data, which fsck leaves in place; it's for the directory entries which fsck has to make up. ) If you accidentally delete lost+found, don't recreate it with mkdir, use mklost+found if available.

UNIX Operating System File System Integrity Checks File systems are usually mounted readonly. This makes it possible to check the file system while it is mounted; it is not a good idea to check a file system that is mounted read-write.

UNIX Operating System Fsck Phases If the superblock is corrupted the file system still can be repaired using alternate superblocks created while making a new file system. The first alternate superblock number is 32 and other superblock numbers can be found using the following command: newfs -N /dev/rdsk/c 0 t 0 d 0 s 6

UNIX Operating System Fsck Phases Video on fsck utility. https: //www. youtube. com/watch? v=nm 9 tjo. OZF-4 https: //www. youtube. com/watch? v=zk 9 df. LQ 2 s. JU

UNIX Operating System /proc/filesystems The /proc/filesystems is a file used to detect file systems supported by the running kernel. If a file system is listed, you will be able to mount it. This text listing of the file system is compiled into the kernel. The “nodev” string in the first column means that file system does not require a block device to be mounted, it’s called a virtual file system.

UNIX Operating System /proc/filesystems It is a special file system in UNIX-like systems that presents information about processes and other system information in a hierarchical file like structure, providing a more convenient and standardized method for dynamically accessing process data held in the kernel than traditional tracing methods or direct access to kernel memory. Typically, it is mapped to a mount point named /proc at boot time.

UNIX Operating System /proc/filesystems The /proc/filesystems file provides a method of communication between kernel space and user space.

UNIX Operating System /proc/filesystems Is a file used to detect filesystems supported by running kernel. You can quickly run grep or the cat command to display the list of all supported file systems. Nodev indicates that the file system is not associated with a physical device such as /dev/sdb 1. If you see ext 3 or vfat, it means you will be able to mount ext 3 and vfat based file systems.

UNIX Operating System /proc/filesystems The Linux implementation of /proc is a clone of Plan 9. Under Linux, /proc includes a directory for each running process (including kernel processes) at /proc/PID, containing information about that process. Plan 9 is an enhancement of the V 8 Plan. Rather than have just one file containing all the information about a process, Plan 9 created a hierarchy of separate files to provide those functions, and made /proc a real part of the file system

UNIX Operating System /proc/filesystems /proc/PID/cmdline, which contains the command that originally started the process. /proc/PID/cwd, a symlink to the current working directory of the process. /proc/PID/environ, a file containing the names and contents of the environment variables that affect the process. /proc/PID/exe, a symlink to the original executable file, if it still exists (a process may continue running after its original executable has been deleted or replaced).

UNIX Operating System /proc/filesystems /proc/PID/fd, a directory containing a symbolic link for each open file descriptor /proc/PID/fdinfo, a directory containing files which describe the position and flags for each open file descriptor. /proc/PID/maps, a text file containing information about mapped files and blocks (like heap and stack).

UNIX Operating System /proc/filesystems /proc/PID/mem, a binary file representing the process's virtual memory, can only be accessed by a ptrace'ing process. /proc/PID/root, a symlink to the root path as seen by the process. For most processes this will be a link to / unless the process is running in a chroot jail. /proc/PID/status, a file containing basic information about a process including its run state and memory usage.

UNIX Operating System /proc/filesystems /proc/PID/task, is a directory containing hard links to many tasks that have been started by this (i. e. : the parent) process.

UNIX Operating System File Representation A file format is a particular way to encode information for storage in a computer file. Disk drives or computer storage are represented in bits. A computer must convert information to 0 s and 1 s and vice-versa. A computer file format can be represented as either a binary image or ASCII file. ASCII is used on most PCs and Macintoshes. EBCDIC (Extended Binary-Coded Decimal Interchange Code is an IBM code for representing characters as numbers. IBM uses EBCDIC widely on their large computers.

UNIX Operating System EBCDIC character set EBCDIC is an 8 bit character encoding used on IBM mainframes and AS/400 s. Single byte EBCDIC takes up eight bits, which are divided in two pieces. The first four bits are called the zone and represent the category of the character, whereas the last four bits are called the digit and identify the specific character.

UNIX Operating System File Representation Binary image files represent 256 different binary codes.

UNIX Operating System Binary 100 0001 100 0010 100 0011 110 0001 110 0010 110 0011 Oct 101 102 103 141 142 143 Dec Hex Char 65 41 A 66 42 B 67 43 C 97 61 a 98 62 b 99 63 c

UNIX Operating System In the past, binary image files were used to represent more character codes than ASCII because eight bit bytes were used, whereas ASCII files used seven bits. Now Extended ASCII uses eight bit bytes and can represent 256 character Executable software, mostly word processing files and databases, spread sheets and multimedia files are binary. However, text and source program files as well as HTML and XML files are ASCII.

UNIX Operating System What does the CMOS do? The CMOS memory stores information your computer needs when it boots up, such as, disk drive, keyboard type, configuration details of the PCI Bus slots, and Video Interface information. If the power to the CMOS memory is lost, your computer will boot with the CMOS defaults. In most cases, this will prevent use of your disk drives until this information is supplied.

UNIX Operating System CMOS specifications The information stored in the CMOS memory is battery backed up. The CMOS memory is a 64 or 128 byte memory that is part of the system clock chipset chips. The CMOS memory is made from Complementary Metal Oxide Semiconductor (CMOS). This material can be backed up by battery, which requires a very low current to keep the data in the memory current when the power is turned off.

UNIX Operating System How is the CMOS information used? The CMOS information is required by your computers Basis Input/Output System (BIOS). The system BIOS is a 64 K by 8 bit wide ROM mounted on the motherboard. If the BIOS ROM were accessed during normal operation the computer would be greatly reduced. Instead, the information is copied to some RAM where it can be accessed 32 or 64 bits at a time, thus increasing the system performance. The differences in access speed between RAM device and ROM’s is approximately 3 times faster.

UNIX Operating System How is the CMOS information used? Read-only memory or ROM is a form of data storage in computers and other electronic devices that can not be easily altered or reprogrammed. RAM is referred to as volatile memory and is lost when the power is turned off whereas ROM in non-volatile and the contents are retained even after the power is switched off.

UNIX Operating System How is the CMOS information used? Random Access Memory (RAM) is a form of data storage that can be accessed randomly at any time, in any order and from any physical location in contrast to other storage devices, such as hard drives, where the physical location of the data determines the time taken to retrieve it. RAM is measured in megabytes and the speed is measured in nanoseconds and RAM chips can read data faster than ROM.

UNIX Operating System

UNIX Operating System Role of system BIOS It is to test the system, initialize some I/O devices, recognize the hardware devices (basic power test), and load the operating system. Once, the operating system is loaded, the BIOS works with the operating system to enable access to the hardware devices. At this point, the BIOS provide basic input/output services to the operating system and applications.

UNIX Operating System

UNIX Operating System

UNIX Operating System

UNIX Operating System What is Unified Extensible Firmware Interface (UEFI)? UEFI is a specification defined between the operating system and the platform firmware. UEFI was developed to replace the Basic I/O System firmware interface system. The UEFI can support remote diagnostics and repair of computers, even with another operating system.

UEFI UNIX Operating System

UEFI UNIX Operating System

UNIX Operating System Initializing or Booting a System 1. CPU is pre-set to RESET mode 2. CPU pre-programmed to seek 0 xfffffff 0 address location 3. BIOS reads MBR (Track 0, Sector 1) 4. First part of MBR contains the boot partition table and executable code 5. Second part of the MBR is used to locate the boot loader. It resides on the “active” partition known as the Boot Sector. 6. Device drivers are loaded (initrd-2. 4. 20 -8. img) 7. Kernel is invoked 8. Init process invoked

UNIX Operating System Linux Boot Procedure 1. The system startup stage depends on the hardware device that Linux is being booted on. Floppy disk, CD-ROM, a partition on a hard disk, a device on the network, or a USB flash memory stick. Linux is usually booted from a hard disk. 2. After the system locates the boot device, the first part of the boot loader is loaded into RAM and executed. Normally, the first part of the boot loader is less than 512 bytes and is responsible for loading the second part of the boot loader.

UNIX Operating System Linux Boot Procedure 3. The first 446 bytes are considered the primary boot loader, which contains both executable code and error message text. 4. The second sixty four bytes are the partition tables, which contains a record for each of the four partitions. 5. The last two bytes are the magic number which serves as a validation check of the Master Boot Record (MBR).

UNIX Operating System Linux Boot Procedure 6. When the second part of the boot loader is loaded into RAM and executed, a pop up screen displays Linux and the optional initial RAM being loaded into memory. 7. The second part of the boot loader is called the kernel loader. After the boot images are loaded, the second part of the boot loader passes control to the kernel image. At this point, the kernel is decompressed and initialized

UNIX Operating System Linux Boot Procedure 8. The second part of the boot loader checks the system hardware, lists attached hardware devices, mounts the root device, and loads the kernel modules.

UNIX Operating System What is the Master Boot Record (MBR) It is a small program that is a special type of boot sector that manages the boot process. The MBR is not located in a partition, but located at the beginning of partitioned computer mass storage devices, such as, fixed disks or removable devices.

UNIX Operating System What is the Master Boot Record (MBR) The organization of the partition table in the Master Boot Record sets the boundaries for the maximum addressable storage space of a disk to 2 TB (2 exp 32 x 512 bytes)

UNIX Operating System Master Boot Record (MBR) When a data storage device has been partitioned with the MBR Partition Table scheme the master boot record contains the primary partition entries in its partition table. By convention, there are exactly four primary partition table entries in the MBR Partition Table scheme.

UNIX Operating System In Linux, and other Unix-like operating systems, the /boot directory holds files used in booting the operating system. vmlinuz is the Linux kernel. initrd. img is a temporary file system, used prior to loading the kernel. system. map is a symbol lookup table.

UNIX Operating System Boot Loaders The following /boot directory contains all the important files which are required for successful booting process: Demonstrate: /boot Red Hat Linux kernels Kernel configuration files for Red Hat Linux Demonstrate: /boot/grub The Linux Loader (LILO) is the combination of parts one and two boot loaders. Because of some disadvantages in the LILO loader, the GRand Unified Bootloader (GRUB) has become more widely used.

lost+found vmlinuz-2. 6. 32 -642. 11. 1. el 6. i 686 vmlinuz-2. 6. 32 -71. 24. 1. el 6. i 686 vmlinuz-2. 6. 32 -71. 29. 1. el 6. i 686

UNIX Operating System Boot loader The role of the boot loader is to place the operating system of the system in memory. In Linux, the most commonly used bootloading tool is LILO, the Linux Loader. During the installation process, Linux will generate LILO values and ask you to verify them. At that point, you are given the opportunity to insert additional LILO options. Such options might be to include more partitions, and identify the operating system to be installed.

UNIX Operating System Boot loader First, what exactly is GRUB? GRUB is a boot loader, which means it passes control of the boot process from the Power-On Self Test (POST) to the kernel of your GNU/Linux distribution. GRUB works in a modular, layered fashion so that any unneeded modules are not loaded. Not only does this reduce execution time, but it saves valuable resources when running from removable media.

UNIX Operating System Boot loader GRUB optionally loads its configuration file at run /boot time, so you don’t have to type in commands manually each time. However, the command-line option is still available in case there is an error in your configuration file.

UNIX Operating System When a computer with Red Hat Enterprise Linux is turned on, the operating system is loaded into memory by a special program called a boot loader. A boot loader usually exists on the system's primary hard drive (or other media device) and has the sole responsibility of loading the Linux kernel with its required files or (in some cases) other operating systems into memory.

UNIX Operating System So why use GRUB when there are other options out there? The beauty of free software is that you have choices. Alternatives to GRUB include LILO, syslinux and isolinux. The benefit of GRUB is that it will work with many different types of boot devices, but you only need to learn one set of menu commands. In addition, GRUB can work on other forms of bootable storage, such as CD-R/W,

UNIX Operating System With the release of Red Hat Enterprise Linux (RHEL) 6, Red Hat will utilize the new Upstart boot service. This service will replace the System V init boot procedure that is based on run levels directories that contained scripts that all had to be started. Upstart is event driven, so it contains scripts that are activated on demand, which makes the boot process a lot faster.

UNIX Operating System In an attempt to simplify the new concept, Upstart still utilizes the services from the init process. The /sbin/init, which is the mother of all user services

UNIX Operating System Init Process Location: /sbin/init First process to execute in user space Executes: background Process ID: 1 (one) Parent Process ID: 0 (zero) Owner: root Highlight the relationship of the init process to all other executing processes.

UNIX Operating System Red Hat Enterprise Release 6. 0 (Upstart) Handles initialization by starting the most fundamental services. /etc/init/rc. S. conf Handles starting the individual runlevels /etc/init/rc. conf The following two areas specify how terminals are to be handled. /etc/init/tty. conf and /etc/init/serial. conf There were some additional configuration files included to the /etc/sysconfig/init file.

UNIX Operating System The /etc/inittab file is referenced only for determining the default run level of the system. All other previous tasks are handled by the new Upstart procedures. The following statement is the only one used in the /etc/inittab file by Red Hat Enterprise 6. 0. id: 3: initdefault

UNIX Operating System UNIX-like runlevels 0 - halt (Do NOT set initdefault to this) 1 - Single user mode 2 - Multiuser, without NFS (The same as 3, if you do not have networking) 3 - Full multiuser mode 4 - unused 5 - X 11 6 - reboot (Do NOT set initdefault to this)

UNIX Operating System UNIX-like performance tools Sar is a command is performance monitoring tool. It can find out what Linux is doing all the time. It can generate report and email them to sys admin. Vmstat is a command that reports information about processes, memory, paging, block IO, traps, and cpu activity. free command displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.

UNIX Operating System Linux performance tools Iostat is a command is used for monitoring system input/output device loading by observing the time the devices are active in relation to their average transfer rates i. e. it is useful to monitor disk throughput.

UNIX Operating System UNIX-like shutdown command The shutdown command is used to shutdown the system in a safe way. After bringing the system down, the shutdown command can halt or reboot the system according to the options that are specified. Example: shutdown –h now

UNIX Operating System /etc/rc. d init. d rc 0. d rc 2. d rc 4. d rc 6. d rc. sysinit rc rc 1. d rc 3. d rc 5. d rc. local /etc rc rc 4. d init. d rc 0. d rc 5. d rc. sysinit rc 1. d rc 6. d rc 2. d rc 3. d rc. local

UNIX Operating System /etc/rc. d/rc 0. d Explain the k and s script prefixes Explain the two digit numbering Knoppix software directory names are similar but slightly different.

UNIX Operating System Roles of the kernel 1. Responsive - users 2. Autonomous - daemons The kernel consists of a collection of low level procedures. These procedures are called system calls and they are the primary means for a program to interact with other parts of the system.

UNIX Operating System The kernel supports the following file systems: Demonstrate: /proc cat filesystems Demonstrate: Execute newuid. pl and process. sh scripts

UNIX Operating System Points of interest MBRs are not present on non-partitioned media like floppies, super floppies or other storage devices.

Moving Around in UNIX uptime w who –r who am i

Break-out problems What functionality does the mingetty or getty perform? Define the kernel responsive and autonomous functions? What system functionality does the /etc/fstab file provide? What functionality does the /etc/sysconfig directory provide? What functionality does the /etc/inittab file provide?

Assignment Announcements Sign up for your open source topic if you have not done so yet. Open source presentations April 3 (week 11) and April 10 (week 12).

After class assistance Questions Comments Concerns I am available after this Franklin Live session to discuss any problems and/or concerns regarding the lab assignments

Lab Assistance available by phone and/or email