L 4 Version 4 Jochen Liedtke Uwe Dannowski

L 4 Version 4 (Jochen Liedtke) Uwe Dannowski Espen Skoglund Volkmar Uhlig © 2001 Universität Karlsruhe, System Architecture Group (X. 2)

About presentation n n Audience should be familiar with L 4 Version 2 and/or Version X. 0 Presentation will focus on: n n n Task/thread creation deletion, and Communication In depth discussions afterwards © 2001 Universität Karlsruhe, System Architecture Group

Major changes from Version 2 n No Clans and Chiefs n n Use “redirection” instead Redesigned IPC API n n Architecture independent API n n Two phases (build and send) n n Optimized for IDL compilers Support for absolute timeouts © 2001 Universität Karlsruhe, System Architecture Group 64 -bit/32 -bit clean Use virtual registers to expand register set (E. g. , for IPC messages) n Portable (Recompilation needed)

Thread IDs n Word sized n n n 32 or 64 bit depending on architecture n Task number not encapsulated in ID n n No restrictions on number of threads in a task (address space) Threads can be migrated between tasks (address spaces) Version numbers not assigned by kernel n Each thread has a global and a local ID n Local ID is only valid inside an address space (Typically location of UTCB) n © 2001 Universität Karlsruhe, System Architecture Group Handled through privileged Thread. Control() syscall Global IDs are unique across all spaces

Privileged threads and system calls n Privileged threads are able to perform certain privileged (“unsafe”) syscalls (e. g. , Memory. Control(), Processor. Control()) n n Only threads within privileged (tasks) address spaces are themselves privileged Only tasks started initially by the kernel are privileged (Sigma 0 and Root Task) © 2001 Universität Karlsruhe, System Architecture Group

Version 4: Task and Thread Creation/Deletion Espen Skoglund © 2001 Universität Karlsruhe, System Architecture Group

Task creation n Three steps required n n n Create empty space Configure newly created space Start initial thread n Two syscalls involved n n n Both syscalls are privileged n © 2001 Universität Karlsruhe, System Architecture Group Space. Control() Thread. Control() Only root task can create/delete tasks

The Space. Control() system call Thread. Id Fpage Thread. Id Word Address spaces have no IDs n A thread ID specifies the space in which the thread resides n Word Space Specifier Kernel Interface Page Area Utcb Area Redirector Control Result The interface page is mapped by the kernel n Argument specifies the virtual memory area of interface page n A nil-page deletes the specified space n (If space only contains one thread) © 2001 Universität Karlsruhe, System Architecture Group

The Space. Control() system call Thread. Id Fpage Thread. Id Word UTCBs are mapped by kernel n Argument specifies virtual memory area of the UTCBs n © 2001 Universität Karlsruhe, System Architecture Group Word Space Specifier Kernel Interface Page Area Utcb Area Redirector Control Result Specifies the “redirector” who controls IPC operations in the address space n Redirectors currently have undefined semantics n

The Space. Control() system call Thread. Id Fpage Thread. Id Word n Controls address space specific settings n n n Word Space Specifier Kernel Interface Page Area Utcb Area Redirector Control Small spaces ASID allocation Not in Schedule() to allow for global allocation schemes © 2001 Universität Karlsruhe, System Architecture Group n Result Indicates whether operation was successful or not

The Thread. Control() system call Thread. Id Identifies addressed thread n May have different version number than existing thread n (Will change version number) n Non-existent threads will be created © 2001 Universität Karlsruhe, System Architecture Group Word Destination Space Specifier Scheduler Pager Result Specfies space in which to create thread n A specifier equal to Destination creates a new empty address space n A nil-thread specifier deletes the addressed thread n

The Thread. Control() system call Thread. Id n Species scheduler for addressed thread Species pager for addressed thread n Upon thread creation: n n n Nil ID creates inactive thread Non-nil ID creates active thread © 2001 Universität Karlsruhe, System Architecture Group Word Destination Space Specifier Scheduler Pager n Result Indicates whether operation was successful or not

Task creation - walkthrough P Thread. Control (A, A, nil) Privileged thread A Thread created inactive Empty address space created (i. e. , without Kernel Interface Page and UTCB area) © 2001 Universität Karlsruhe, System Architecture Group

Task creation - walkthrough P Space. Control (A, ~, ~, ~) A Thread is still inactive Address space now contains Kernel Interface Page and UTCB area © 2001 Universität Karlsruhe, System Architecture Group

Task creation - walkthrough P Thread. Control (A, A, SA, PA) A Thread is now active How does A know where to start executing? © 2001 Universität Karlsruhe, System Architecture Group

Thread startup protocol n n Actively created threads start receiving a startup message from their pager Pager sends IP and SP to thread SP MR MR IP Kernel sets IP and MR ~ 0 0 SP according to Startup message received from pager startup message The initial thread starts executing 2 n 1 0 n © 2001 Universität Karlsruhe, System Architecture Group 2

Task creation - walkthrough P A Recv (PA, ~, ~) Send (A, ip, sp) PA © 2001 Universität Karlsruhe, System Architecture Group

Task deletion n Delete all but one thread from address space n n n Address space need not be completely deleted n Thread. Control (t, nil, . . . ) Delete address space using remaining thread n n Space. Control (t, nil, . . . ) Safe since only privileged threads can create new threads inside address space © 2001 Universität Karlsruhe, System Architecture Group n n Leave remaining thread as inactive Flush complete address space Allows for cheap task creation (only one Thread. Control() call to active initial thread)

The Exchange. Registers() system call Thread. Id Word Thread. Id Word Destination Control SP IP FLAGS Pager User Defined Handle Addressed thread n Must be located in same address space n May use global or local ID n Thread. Id Word Thread. Id Word n Specifies what to do n n n © 2001 Universität Karlsruhe, System Architecture Group Result Control SP IP FLAGS Pager User Defined Handle Halt/resume thread? Abort send and/or receive IPC operations? Set any of the SP, IP, FLAGS, Pager, or User Defined Handle values of addressed thread?

The Exchange. Registers() system call Thread. Id Word Thread. Id Word Destination Control SP IP FLAGS Pager User Defined Handle Specifes new SP, IP, FLAGS, and Pager of addressed thread n Ignored if the respective flag in Control is not set n © 2001 Universität Karlsruhe, System Architecture Group Thread. Id Word Thread. Id Word n Result Control SP IP FLAGS Pager User Defined Handle Specifies new User Defined Handle n n Value ignored by kernel Can be used by, e. g. , userlevel thread packages

The Exchange. Registers() system call Thread. Id Word Thread. Id Word n Destination Control SP IP FLAGS Pager User Defined Handle Addressed thread’s global or local ID n Depending on whether Destination was global or local Thread. Id Word Thread. Id Word n n Old values of SP, IP, etc. Current state of addressed thread n n © 2001 Universität Karlsruhe, System Architecture Group Result Control SP IP FLAGS Pager User Defined Handle Halted or not? Within IPC send or receive phase?

Thread creation - two alternatives n n Create new threads using Thread. Control() Alternative A: n n Generate a pool of new threads Each thread is halted upon startup (using Exchange. Registers()) n Threads can be “created” locally from the pool (using Exchange. Registers() to resume them) n Alternative B: n n No pool is generated Each thread creation must happen externally © 2001 Universität Karlsruhe, System Architecture Group

Version 4: Communication Espen Skoglund © 2001 Universität Karlsruhe, System Architecture Group

Version 4 IPC mechanism IPC operations happen in two steps: n Message construction n n Specify message contents (64 words) Message sending n Invoke IPC system call © 2001 Universität Karlsruhe, System Architecture Group

Message construction n Messages are stored in registers (MR 0 MR 63) n n n Message registers are backed by real registers or memory First register (MR 0) acts as message tag Subsequent registers contain: n n Untyped words (u), and Typed words (t) (e. g. , map item, string item) © 2001 Universität Karlsruhe, System Architecture Group Number of untyped words Number of typed words Various IPC flags MR 0 label flags t u Message Tag Freely available (e. g. , request type)

Message construction n Messages are stored in registers (MR 0 MR 63) n n n Message registers are backed by real registers or memory First register (MR 0) acts as message tag Subsequent registers contain: n n Untyped words (u), and Typed words (t) (e. g. , map item, string item) © 2001 Universität Karlsruhe, System Architecture Group MR 8 MR 7 MR 6 MR 5 MR 4 MR 3 MR 2 MR 1 MR 0 label flags Message 5 t 3 u

Message construction n n Typed items occupy one or more words MR Three currently defined MR MR items: n n Map item (2 words) Grant item (2 words) String item (2+ words) Typed items can have arbitrary order © 2001 Universität Karlsruhe, System Architecture Group 8 String Item 7 6 MR 5 MR 4 Map Item MR 3 MR 2 MR 1 MR 0 label flags Message 5 t 3 u

Map and Grant items n Two words: n Send base n n send base MRi+1 0 100 C MR i Map Item Fpage n n n Similar to Version 2 send fpage Similar to Version 2 Contains more access bits Lower bits of send base indicates map or grant item © 2001 Universität Karlsruhe, System Architecture Group send fpage send base MRi+1 0 101 C MRi Grant Item size 0 wrx location Fpage

String items n n n No alignment required Max size 4 MB (per string) Compound strings supported n n Allows scatter-gather Incorporates cacheability hints n Reduce cache pollution for long copy operations © 2001 Universität Karlsruhe, System Architecture Group string pointer string length String Item “hh” indicates cacheability hints for the string MRi+1 0 0 0 hh. C MRi

String items New string specifier may of course contain substrings Different size compound strings require a new string specifier All substrings are of same size string pointer MRi+j+k string pointer MRi+j+3 string pointer MRi+j+2 string length string pointer MRi+j string pointer MRi+2 string pointer MRi+1 string length String Item “hh” indicates cacheability hints for the string © 2001 Universität Karlsruhe, System Architecture Group 0 k 0 - 1 0 hh. C MRi+j+1 0 j -0 1 0 hh. C MRi 1

Message sending n n The Ipc() syscall will transfer message registers from sender to receiver Strings will be copied to receiver (if receiver has specified sufficiently large receive buffers) n Mappings will be mapped/granted (if receiver has specified an appropriate receive window) n No page-faults will occur if no strings are copied n n No page-faults on accessing message registers “Short IPCs” are thus up to 64 words © 2001 Universität Karlsruhe, System Architecture Group

The Ipc() system call Thread. Id Word To From Specifier Timeouts Receiver of message n Nil if no send-phase included n Thread. Id n Specifies who to receive from: n n n Different To and From specifiers allows for, e. g, a combined send reply and wait for other thread! © 2001 Universität Karlsruhe, System Architecture Group From n Nil indicates no receive-phase Anythread indicates an open receive Anylocalthread indicates an open receive from a local thread Any other thread ID

The Ipc() system call Thread. Id Word n Specifies Send and Receive timeouts n n To From Specifier Timeouts Timeout before IPC operation initiates Transfer timeout specified in a separate virtual register n Combined timeout of all page faults during IPC string copy © 2001 Universität Karlsruhe, System Architecture Group Thread. Id n From Indicates sender of message (only for IPCs with receive phase) Result of IPC is stored in MR 0 n Received untyped words and typed items are stored in remaining MRs n

The Lipc() system call (local IPC) n n Identical semantics as normal Ipc() Lipc() has some restrictions n n n Sender and receiver are in same address space Sender and receiver runs on same processor Includes both send and receive phase Receive timeout is (infinite) IPC includes no map/grant operation © 2001 Universität Karlsruhe, System Architecture Group n Lipc() can fallback to normal Ipc() n n Enables fast local IPC implementation, useful for, e. g. : n n n If restrictions are not met Distributing requests Intra-address-space synchronization Lipc() not necessarily needed for all architectures

Receiving messages n Receiver buffers are specified in registers (BR 0 … BR 33) n n Backed by real registers or memory First BR (BR 0) contains “Acceptor” n n May specify receive window (if not nil-fpage) May indicate presence of receive strings/buffers (if s-bit set) © 2001 Universität Karlsruhe, System Architecture Group receive window Acceptor 000 s BR 0

Receiving messages If C-bit in string item is cleared, it indicates that no more receive buffers are present A receive buffer can of course be a compound string If C-bit in string item is set, it indicates presence of more receive buffers string pointer BR 4+j string pointer BR 5 string pointer BR 4 string length string pointer string length receive window The s-bit set indicates presence of string items acting as receive buffers © 2001 Universität Karlsruhe, System Architecture Group 0 j -0 1 0 hh. C 0 BR 3 Acceptor BR 2 0 0 0 hh. C 1 BR 1 000 s BR 0

The Unmap() system call Word Void Control n Specifies unmap or flush, and n How many fpages to unmap n n Fpages are listed in MRs Each fpage indicate permissions to remove © 2001 Universität Karlsruhe, System Architecture Group Each fpage (MR) will on return contain page access bits n n Written Referenced e. Xecuted Page access bits are reset upon unmap n A no revocation unmap will just reset and return previous access bits n