Windows Kernel Internals Process Architecture David B Probert

Windows Kernel Internals Process Architecture *David B. Probert, Ph. D. Windows Kernel Development Microsoft Corporation © Microsoft Corporation 1

Process/Thread structure Any Handle Table Object Manager Process Object Thread Files Events Process’ Handle Table Virtual Address Descriptors Devices Thread Drivers Thread © Microsoft Corporation 2

Process • Container for an address space and threads • Primary Token • Quota, Debug port, Handle Table etc • Unique process ID • Queued to the Job, global process list and Session list • MM structures like the VAD tree, AWE etc © Microsoft Corporation 3

Thread • Fundamental schedulable entity in the system • Structure is the ETHREAD that holds a KTHREAD • Queued to the process (both E and K thread) • IRP list • Impersonation information • Unique thread ID • Flags or various sorts and TEB pointer © Microsoft Corporation 4

Job • Container for multiple processes • Queued to global job list, processes and jobs in the job set • Security token filters and job token • Completion ports • Counters, limits etc © Microsoft Corporation 5

KPROCESS fields DISPATCHER_HEADER Header ULPTR Directory. Table. Base[2] KGDTENTRY Ldt. Descriptor KIDTENTRY Int 21 Descriptor USHORT Iopm. Offset UCHAR Iopl volatile KAFFINITY Active. Processors ULONG Kernel. Time ULONG User. Time LIST_ENTRY Ready. List. Head SINGLE_LIST_ENTRY Swap. List. Entry LIST_ENTRY Thread. List. Head KSPIN_LOCK Process. Lock KAFFINITY Affinity USHORT Stack. Count SCHAR Base. Priority SCHAR Thread. Quantum BOOLEAN Auto. Alignment UCHAR State BOOLEAN Disable. Boost UCHAR Power. State BOOLEAN Disable. Quantum UCHAR Ideal. Node © Microsoft Corporation 6

EPROCESS fields KPROCESS Pcb EX_PUSH_LOCK Process. Lock LARGE_INTEGER Create. Time LARGE_INTEGER Exit. Time EX_RUNDOWN_REF Rundown. Protect HANDLE Unique. Process. Id LIST_ENTRY Active. Process. Links Quota Felds SIZE_T Peak. Virtual. Size SIZE_T Virtual. Size LIST_ENTRY Session. Process. Links PVOID Debug. Port PVOID Exception. Port PHANDLE_TABLE Object. Table EX_FAST_REF Token PFN_NUMBER Working. Set. Page KGUARDED_MUTEX Address. Creation. Lock KSPIN_LOCK Hyper. Space. Lock struct _ETHREAD *Fork. In. Progress ULONG_PTR Hardware. Trigger; PMM_AVL_TABLE Physical. Vad. Root PVOID Clone. Root PFN_NUMBER Number. Of. Private. Pages PFN_NUMBER Number. Of. Locked. Pages PVOID Win 32 Process struct _EJOB *Job PVOID Section. Object PVOID Section. Base. Address PEPROCESS_QUOTA_BLOCK Quota. Block © Microsoft Corporation 7

EPROCESS fields PPAGEFAULT_HISTORY Working. Set. Watch HANDLE Win 32 Window. Station HANDLE Inherited. From. Unique. Process. Id PVOID Ldt. Information PVOID Vad. Free. Hint PVOID Vdm. Objects PVOID Device. Map PVOID Session UCHAR Image. File. Name[ 16 ] LIST_ENTRY Job. Links PVOID Locked. Pages. List LIST_ENTRY Thread. List. Head ULONG Active. Threads PPEB Peb IO Counters PVOID Awe. Info MMSUPPORT Vm Process Flags NTSTATUS Exit. Status UCHAR Priority. Class MM_AVL_TABLE Vad. Root © Microsoft Corporation 8

KTHREAD fields DISPATCHER_HEADER Header LIST_ENTRY Mutant. List. Head PVOID Initial. Stack, Stack. Limit PVOID Kernel. Stack KSPIN_LOCK Thread. Lock ULONG Context. Switches volatile UCHAR State KIRQL Wait. Irql KPROC_MODE Wait. Mode PVOID Teb KAPC_STATE Apc. State KSPIN_LOCK Apc. Queue. Lock LONG_PTR Wait. Status PRKWAIT_BLOCK Wait. Block. List BOOLEAN Alertable, Wait. Next UCHAR Wait. Reason SCHAR Priority UCHAR Enable. Stack. Swap volatile UCHAR Swap. Busy LIST_ENTRY Wait. List. Entry NEXT Swap. List. Entry PRKQUEUE Queue ULONG Wait. Time SHORT Kernel. Apc. Disable SHORT Special. Apc. Disable KTIMER Timer KWAIT_BLOCK Wait. Block[N+1] LIST_ENTRY Queue. List. Entry UCHAR Apc. State. Index BOOLEAN Apc. Queueable BOOLEAN Preempted BOOLEAN Process. Ready. Queue BOOLEAN Kernel. Stack. Resident © Microsoft Corporation 9

KTHREAD fields cont. UCHAR Ideal. Processor volatile UCHAR Next. Processor SCHAR Base. Priority SCHAR Priority. Decrement SCHAR Quantum BOOLEAN System. Affinity. Active CCHAR Previous. Mode UCHAR Resource. Index UCHAR Disable. Boost KAFFINITY User. Affinity PKPROCESS Process KAFFINITY Affinity PVOID Service. Table PKAPC_STATE Apc. State. Ptr[2] KAPC_STATE Saved. Apc. State PVOID Callback. Stack PVOID Win 32 Thread PKTRAP_FRAME Trap. Frame ULONG Kernel. Time, User. Time PVOID Stack. Base KAPC Suspend. Apc KSEMAPHORE Suspend. Sema PVOID Tls. Array LIST_ENTRY Thread. List. Entry UCHAR Large. Stack UCHAR Power. State UCHAR Iopl CCHAR Freeze. Cnt, Suspend. Cnt UCHAR User. Ideal. Proc volatile UCHAR Deferred. Proc UCHAR Adjust. Reason SCHAR Adjust. Increment © Microsoft Corporation 10

ETHREAD fields KTHREAD tcb Timestamps LPC locks and links CLIENT_ID Cid Impersonation. Info Irp. List p. Process Start. Address Win 32 Start. Address Thread. List. Entry Rundown. Protect Thread. Push. Lock © Microsoft Corporation 11

Thread and Process Enumeration • Threads and processes all enumerable until their last reference is released • No need to hold locks while processing each process/thread • Code uses safe references to prevent the double return to zero problem © Microsoft Corporation 12

Thread Enumeration Example for (Thread = Ps. Get. Next. Process. Thread (Process, NULL); Thread != NULL; Thread = Ps. Get. Next. Process. Thread (Process, Thread)) { st = STATUS_SUCCESS; if (Thread != Self) { Psp. Terminate. Thread. By. Pointer (Thread, Exit. Status); } } © Microsoft Corporation 13

Process Enumeration Internals PEPROCESS Ps. Get. Next. Process (IN PEPROCESS Process) { for (List. Entry = Process->Active. Process. Links. Flink; List. Entry != &Ps. Active. Process. Head; List. Entry = List. Entry->Flink) { New. Process = CONTAINING_RECORD (List. Entry, EPROCESS, Active. Process. Links); if (Ob. Reference. Object. Safe (New. Process)) { break; } New. Process = NULL; } © Microsoft Corporation 14

Process Creation BOOL WINAPI Create. Process. W( LPCWSTR lp. Application. Name, LPWSTR lp. Command. Line, LPSECURITY_ATTRIBUTES lp. Process. Attributes, LPSECURITY_ATTRIBUTES lp. Thread. Attributes, BOOL b. Inherit. Handles, DWORD dw. Creation. Flags, LPVOID lp. Environment, LPCWSTR lp. Current. Directory, LPSTARTUPINFOW lp. Startup. Info, LPPROCESS_INFORMATION lp. Process. Information ) © Microsoft Corporation 15

Process Creation – Create. Process() Locate imagefile (path search) Convert DOS name to NT name Call Nt. Open. File() Call Nt. Create. Section(SEC_IMAGE) Check for special handling: VDM, Wo. W 64, restrictions, CMD files Call Nt. Query. Section() to get Image. Information Use Ldr. Query. Image. File. Execution. Options() to see if debugging Special handling for POSIX executable Create the new process in the kernel via Nt. Create. Process. Ex() If requested, call Nt. Set. Information. Process(Process. Priority. Class) If (dw. Creation. Flags & CREATE_DEFAULT_ERROR_MODE) call Nt. Set. Information. Process(Process. Default. Hard. Error. Mode) © Microsoft Corporation 16

Create. Process() – cont. Call Base. Push. Process. Parameters() to push params into new process Stuff in the standard handles if needed Call Base. Create. Stack() to create a user-mode stack in process Call Base. Initialize. Context() to create an initial thread context Call Nt. Create. Thread() to create the first thread // thread may run, so no more modification to new process virtual space Use Csr. Client. Call. Server(Basep. Create. Process) to register new process and thread with CSRSS If app is restricted Set a restricted token on the process assign it to a job object so that it can't escape the token. Unless the initial thread was created suspended, start it with Nt. Resume. Thread() © Microsoft Corporation 17

Nt. Resume. Thread() Acquire thread's Apc. Queue. Lock and raise to Synch Level Decrement the Suspend. Count If Suspend. Count and Freeze. Count both 0 Lock the dispatcher database Increment the thread's Suspend. Semaphore and call Ki. Wait. Test() to resume thread Unlock the dispatcher database Release thread's Apc. Queue. Lock Call Ki. Exit. Dispatcher(), which may schedule a new thread © Microsoft Corporation 18

Base. Create. Stack(Process, [Stack. Size], [Max. Stack. Size], p. Initial. Teb) If not specified, fill Stack. Size and Max. Stack. Size out of image header, check PEB for minimum Stack. Size Use Nt. Allocate. Virtual. Memory (&Stack, Max. Stack. Size, MEM_RESERVE) to reserve the usermode stack Remember Base/Limit of stack in the TEB Stack. Top = Stack + Max. Stack. Size - Stack. Size Commit stack: Nt. Allocate. Virtual. Memory(Stack. Top, Stack. Size, MEM_COMMIT) If there is room (Stack. Top > Stack), create a guard page: Nt. Protect. Virtual. Memory(Stack. Top - PAGE_SIZE, PAGE_GUARD) © Microsoft Corporation 19

Base. Push. Process. Parameters() Base. Push. Process. Parameters( dw. Base. Push. Process. Parameters. Flags, Process. Handle, Peb, lp. Application. Name, Curdir. Buffer, Quote. Insert || Quote. Cmd. Line ? Quoted. Buffer : lp. Command. Line, lp. Environment, &Startup. Info, dw. Creation. Flags | dw. No. Window, b. Inherit. Handles, Is. Wow. Binary ? IMAGE_SUBSYSTEM_WINDOWS_GUI : 0, p. App. Compat. Data, cb. App. Compat. Data ) © Microsoft Corporation 20

Base. Push. Process. Parameters(newproc) Build up the DLL and EXE search paths, the Command. Line. String, Current. Dir. String, Desktop. Info, and Window. Title Call Rtl. Create. Process. Parameters() to put them into a RTL_USER_PROCESS_PARAMETERS buffer Call Nt. Allocate. Virtual. Memory(newproc) for the environment block Call Nt. Write. Virtual. Memory(newproc) to copy the environment block Finish filling in the Process. Parameter. Block Copy in more of the main window settings Set the console handles for stdin/stdout/stderr Set PROFILE flags Call Nt. Allocate. Virtual. Memory(newproc) for Process. Parameter. Block Copy in with Nt. Write. Virtual. Memory(newproc) Modify the PEB in newproc so that it points to the parameter block Allocate and write App. Compat data to the new process Set pointer in new process’ PEB © Microsoft Corporation 21

Rtl. Create. Process. Parameters() Formats NT style RTL_USER_PROCESS_PARAMETERS record Record self-contained in block of memory allocated by this function Allocation method is opaque so free with Rtl. Destroy. Process. Parameters The process parameters record is created in a de-normalized form Caller will fill in additional fields before calling Rtl. Create. User. Process() © Microsoft Corporation 22

Kernel: Nt. Create. Process. Ex() Take reference on parent process, if specified Create an object of Ps. Process. Type for KPROCESS/EPROCESS object Initialize rundown protection in the thread Call Psp. Inherit. Quota() to set the quota block Call Ob. Inherit. Device. Map() to setup Dos. Devices to right device map If passed section handle, take reference -- otherwise clone parent VA If cloning parent, acquire rundown protection to avoid parent exit If passed debug and/or exception ports, point newproc at them Call Mm. Create. Process. Address. Space() If not cloning a parent Process->Object. Table = Current. Process->Object. Table Call Ke. Initialize. Process() to init newproc with default scheduling information and mark newproc as In. Memory Call Psp. Initialize. Process. Security() to duplicate the parents token as the primary token for the process Initialize the fast references for newproc’s token Set newproc’s scheduling parameters If cloning a parent Call Ob. Init. Process() © Microsoft Corporation 23

Nt. Create. Process. Ex() – cont. // Initialize newproc’s address space. Four possibilities Boot Process: Address space already initialized by Mm. Init() System Process: Address space only maps system space (process is same as Psp. Initial. System. Process) Cloned User Process: Address space cloned from specified parent New User Process: Address is initialized to map specified section If cloning parent Call Mm. Initialize. Process. Address. Space(Process, Parent) else Call Mm. Initialize. Process. Address. Space(Process, Section. Object) Call Ex. Create. Handle(Psp. Cid. Table) to allocate a CID for the process Set the process CID in the handle table (for checks and debugging) If parent in a job add in this process to the job If cloning parent Call Mm. Create. Peb() else Copy the parents PEB via Mm. Copy. Virtual. Memory() © Microsoft Corporation 24

Nt. Create. Process. Ex() – cont. 2 Insert new process into the global process list (Ps. Active. Process. Head) Call Se. Create. Access. State. Ex() to create an Access. State structure Call Ob. Insert. Object(Process, Access. State, Desired. Access, &handle) into the handle table Write the handle back into the user-mode handle buffer Call Ob. Get. Object. Security (Process, &Security. Descriptor) and pass to Se. Access. Check() If the access check fails, take away all process access rights Call Ke. Query. System. Time (&Process->Create. Time) Give back the extra reference we used to keep the process from being prematurely deleted © Microsoft Corporation 25

Nt. Create. Section(SEC_IMAGE) Validate/capture parameters and call Mm. Create. Section() Call Cc. Wait. For. Uninitialize. Cache. Map() to synch with teardown of residual data section refs in cache manager Allocate a temporary Control. Area Acquire the ERESOURCE lock to synchronize with the file system Call Mi. Find. Image. Section. Object() to find an existing image Control. Area for this file Call Mi. Lock. Pfn. Database() to take PFN lock Deal with race conditions, like existing Control. Area being deleted Call Mi. Unlock. Pfn. Database() to release PFN lock © Microsoft Corporation 26

Nt. Create. Section (SEC_IMAGE) – 2 If existing Control. Area New Section. Object will share the segment in the existing Control. Area, so Number. Of. Section. References++ Call Mi. Flush. Data. Section() to flush any data section for the file Discard the temporary Control. Area Release the ERESOURCE file system lock else Use the temporary Control. Area we allocated Call Mi. Insert. Image. Section. Object(File, Control. Area) to insert the new Control. Area into the File. Object Call Mi. Create. Image. File. Map(File, &Segment) to do the actual mapping and create real Control. Area Call Ke. Acquire. Queued. Spin. Lock(Lock. Queue. Pfn. Lock) Call Mi. Remove. Image. Section. Object (File, New. Control. Area) Call Mi. Insert. Image. Section. Object (File, real Control. Area) Delete the temporary Control. Area Deal with race conditions, like another thread creating the same section Call Ke. Release. Queued. Spin. Lock(Lock. Queue. Pfn. Lock) © Microsoft Corporation 27

Nt. Create. Section (SEC_IMAGE) – 3 Call Ob. Create. Object (Mm. Section. Object. Type, &New. Section. Object) to create the real section object Fill in New. Section. Object with the values we have accumulated on our stack Pass out the New. Section. Object Ob. Insert. Object(Section, . . . , &handle) © Microsoft Corporation 28

Mi. Find. Image. Section. Object() Searches the control area chains (if any) for an existing cache of the specified image file For non-global control areas, there is no chain and control area is shared for all callers and sessions Likewise for systemwide global control areas For global PER-SESSION control areas, we must walk the list Mi. Insert. Image. Section. Object() Inserts the control area into the file's section object pointers For non-global control areas and systemwide, there is no chain … For global PER-SESSION control areas, we must do a list insertion © Microsoft Corporation 29

Mi. Create. Image. File. Map() Call Fs. Rtl. Get. File. Size(File, &End. Of. File) Read in the image header and validate it: Initialize an Event and an Mdl on the stack Call Mi. Get. Page. For. Header() to allocate pageframe for image header Call Mi. Flush. Data. Section() Call Io. Page. Read(File, Mdl, 0, Event) to do the read Wait on the Event Call Mi. Map. Image. Header. In. Hyper. Space() to map the image header into per-process KVA Validate image header If header more than one page, read another 8 KB Compute the number of PTEs needed to map the image Allocate a control area and a subsection for each section header plus one for the image header which has no section Establish the prototype PTEs for each subsection, and point them all at their subsection Return the Segment © Microsoft Corporation 30

Mm. Create. Process. Address. Space (x 86) Take the Working. Set lock Take the PFN lock so we can get physical pages Allocate the page directory and set into Directory. Table. Base[0] Allocate the page directory for hyperspace and set into Directory. Table. Base[1] Allocate pages for the VAD allocation bitmap and the working set list Release the PFN lock Initialize the hyperspace map Under the expansion lock insert the new process onto MM's internal Process. List Map the page directory page into hyperspace Setup the self-map Fill in the system page directories Release the Working. Set lock Increment the session reference count © Microsoft Corporation 31

Mm. Create. Peb() Attach to the target process Map in the NLS tables Call Mi. Create. Peb. Or. Teb() to allocate a PEB in the user address space Initialize the PEB, including values from the Initial. Peb, the NLS tables, the system defaults, and the image header Detach from the process Return the allocated PEB address Mi. Create. Peb. Or. Teb() Allocate VAD and mark non-deletable and with unchangeable protection Lock the address space Find a VA for the block Finish initializing the VAD Unlock the address space © Microsoft Corporation 32

Nt. Create. Thread() Take a reference on the process that will contain the thread Create an object of Ps. Thread. Type (this will contain the KTHREAD/ETHREAD data structure) Initialize the rundown protection in the thread Point the thread at its process Initialize the various fields used by MM, LPC, IO, Registry, thread lock, timers, queues, etc. Call Ex. Acquire. Rundown. Protection() to keep the process from terminating (bail if it is already doing so) Call Mm. Create. Teb() to create the user-mode TEB Set the Start. Address and Win 32 Start. Address in the kernel thread object Call Ke. Init. Thread() to finish setting up the thread object N. B. kernel-mode execution will begin at Psp. User. Thread. Startup © Microsoft Corporation 33

Nt. Create. Thread() – cont. Take the process lock: Psp. Lock. Process. Exclusive() Process->Active. Threads++ Insert thead at tail of Process->Thread. List Call Ke. Start. Thread() to set up thread Call Psp. Unlock. Process. Exclusive() Call Ex. Release. Rundown. Protection() If this is the first thread in the process invoke callbacks registered for notification of process creation If process is in a job and this is our first chance to report in, send the notification to the job's Completion. Port Invoke callbacks for notification of thread creation If thread was to be created suspended, call Ke. Suspend. Thread() on it Call Se. Create. Access. State. Ex() to create an Access. State structure Call Ob. Insert. Object(Thread, Access. State, Desired. Access, &handle) into the handle table Write the handle back into the user-mode handle buffer © Microsoft Corporation 34

Nt. Create. Thread() – cont. 2 Set the thread Create. Time Call Ob. Get. Object. Security (Thread, &Security. Descriptor) and pass to Se. Access. Check() If the access check fails, take away all access to the thread except terminate, set/query information Call Ke. Ready. Thread() Give back the extra reference we used to keep the thread from being prematurely deleted © Microsoft Corporation 35

Csr. Client. Call. Server (Basep. Create. Process) Acquire. Process. Structure. Lock() Duplicate handles to the process and thread into CSRSS Allocate a process structure within CSRSS Copy any per-process data from parent structure to child structure Set CSRSS' Csr. Api. Port to be the child's exception port If the process being debugged, setup debug port and the process group, if we are the leader. Capture thread creation time as a sequence number for the tid Allocate a thread structure within CSRSS Increment process Thread. Count, insert thread into process Thread. List Insert thread into Csr. Thread. Hash. Table[] Bump reference count on current session Write the pid/tid into process and thread structures Save the duplicated process/thread handles in their respective structures Add the process to the tail of the global list For each DLL loaded in CSRSS notify it about the new process Tell the kernel that the new process is a background process Release. Process. Structure. Lock() © Microsoft Corporation 36

Ke. Init. Thread() The priority, affinity, and initial quantum are taken from the parent process object Initialize most the other fields including the thread context Thread->State = Initialized Set intial code to run: Psp. User. Thread. Startup() Call Ki. Initialize. User. Apc() to set an initial user-mode APC to the thread Initial APC will execute Ldr. Initialize. Thunk() © Microsoft Corporation 37

Ke. Start. Thread() Initialize some more fields (Disable. Boost, Iopl, Quantum, . . . ) Raise to SYNC_LEVEL and acquire Process. Lock Copy the Base. Priority and Affinity from the process Set the Ideal. Processor Lock the dispatcher database Insert thread into process list and increment process Stack. Count Unlock the dispatcher database Lower the IRQL and release Process. Lock © Microsoft Corporation 38

Ldr. Initialize()/Ldrp. Initialize() // Ldrp. Process. Initialized // 0 means no thread has been tasked to initialize the process // 1 means a thread has been tasked but has not yet finished // 2 means a thread has been tasked and initialization is complete while (1 == Interlock. Comp. Exch (&Ldrp. Process. Initialized, 1, 0)) while (Ldrp. Process. Initialized == 1) Nt. Delay. Execution(30 m. S) If Ldrp. Process. Initialized == 0 Initialize the Loader. Lock Call Ldrp. Initialize. Process() Ldrp. Touch. Thread. Stack (Peb->Minimum. Stack. Commit) Interlocked. Increment (&Ldrp. Process. Initialized) // 1 -> 2 else if (Peb->Inherited. Address. Space) Initialize critical section list // otherwise don’t clobber the clone else Call Ldrp. Initialize. Thread() © Microsoft Corporation 39

Ldrp. Initialize. Process() Figure out the image name from the Process. Parameters Nt. Header = Rtl. Image. Nt. Header(Peb->Image. Base. Address) Check Image. File. Execution. Options for this image in the registry Process. Parameters = Rtl. Normalize. Process. Params (Peb->Process. Parameters) Rtl. Init. Nls. Tables (Peb->Ansi. Code. Page. Data, Peb->Oem. Code. Page. Data, Peb->Unicode. Case. Table. Data, &x. Init. Table. Info) Setup process parameters based on the image file Initialize process data structures for allocation TLS and FLS Initialize the Loader. Lock Initialize various critical sections Call Rtl. Initialize. Heap. Manager() Process. Heap = Rtl. Create. Heap() Ldrp. Heap = Rtl. Create. Heap() Call Rtl. Initialize. Atom. Package() Setup DLL search path and current directory from Process. Parameters © Microsoft Corporation 40

Ldrp. Initialize. Process() – cont. Initialize the loaded module list and insert the image into the list If this is a Windows GUI app, load Call Ldr. Load. Dll(kernel 32. dll) Call Ldrp. Walk. Import. Descriptor() to recursively walk the Import Descriptor Table (IDT) and load each referenced DLL If the image was not loaded at the base address in the binary, toggle page protections and call Ldr. Relocate. Image() Call Ldrp. Initialize. Tls() Now that all DLLs are loaded, if (Peb->Being. Debugged) Call Dbg. Break. Point() to notify the debugger Load App. Compat shim engine and shims Call Ldrp. Run. Initialize. Routines() to run all the DLL initialization routines © Microsoft Corporation 41

Ldrp. Initialize. Thread() Take the Loader. Lock Walk the loaded module list calling the DLL init routines: Ldrp. Call. Init. Routine(DLL_THREAD_ATTACH) If the image has TLS, call its initializaers: Ldrp. Call. Tls. Initializers(DLL_THREAD_ATTACH) Release the Loader. Lock © Microsoft Corporation 42

Synchronization Classes • Write once fields like process job and thread impersonation info • Torn down (rundown) structures like handle table, thread TEB etc • Infrequently changing fields like the process token • Frequently changing stuff like thread list of a process or impersonation token © Microsoft Corporation 43

Process Synchronization • Process. Lock – Protects thread list, token • Rundown. Protect – Cross process address space, image section and handle table references • Token, Prefetch – Uses fast referencing • AWE – Uses cache aware pushlocks • Token, Job – Torn down at last process dereference without synchronization © Microsoft Corporation 44

Thread scheduling states © Microsoft Corporation 45

Thread scheduling states • Main quasi-states: – Ready – able to run – Running – current thread on a processor – Waiting – waiting an event • For scalability Ready is three real states: – Deferred. Ready – queued on any processor – Standby – will be imminently start Running – Ready – queue on target processor by priority • Goal is granular locking of thread priority queues • Red states related to swapped stacks and © Microsoft Corporation processes 46

Process Lifetime • Created as an empty shell • Address space created with only ntdll and the main image unless forked • Handle table created empty or populated via duplication from parent • Process is partially destroyed on last thread exit • Process totally destroyed on last dereference © Microsoft Corporation 47

Thread Lifetime • Created within a process with a CONTEXT record • Starts running in the kernel but has a trap frame to return to use mode • Kernel queues user APC to do ntdll initialization • Terminated by a thread calling Nt. Terminate. Thread/Process © Microsoft Corporation 48

Nt. Terminate. Thread(thandle, status) Psp. Terminate. Thread. By. Pointer(p. Thread, status, b. Self) if (b. Self) Psp. Exit. Thread(status) // never returns if Thread->Cross. Thread. Flags & _TERMINATED return exit. Apc = Ex. Allocate. Pool(sizeof(KAPC)) Ke. Initialize. Apc (Exit. Apc, Thread, Original. Apc. Environment, // thread has to detach before exiting Ps. Exit. Special. Apc, Psp. Exit. Apc. Rundown, // runs at end to free exit. Apc Psp. Exit. Normal. Apc, Kernel. Mode, status) Ke. Insert. Queue. Apc (Exit. Apc, NULL, 2) Ke. Force. Resume. Thread (&Thread->Tcb) © Microsoft Corporation 49

Psp. Exit. Thread(status) Ex. Wait. For. Rundown. Protection. Release (&Thread->Rundown. Protect) <Notify registered callout routines of thread exit> Psp. Lock. Process. Exclusive (Process, Thread) Process->Active. Threads-if (Process->Active. Threads == 0) Last. Thread = TRUE Process->Flags |= PROCESS_DELETE Wait until all other threads have exited Psp. Unlock. Process. Exclusive (Process, Thread) if (Process->Debug. Port) Last. Thread? Dbgk. Exit. Process (status) : Dbgk. Exit. Thread (status) // rundown Win 32 (Psp. W 32 Thread. Callout) (Thread, Ps. W 32 Thread. Callout. Exit) if (Last. Thread) (Psp. W 32 Process. Callout) (Process) © Microsoft Corporation 50

Psp. Exit. Thread(status) cont. 1 Io. Cancel. Thread. Io (Thread) Ex. Timer. Rundown () Cm. Notify. Run. Down (Thread) Ke. Rundown. Thread () <Delete thread's TEB> Lpc. Exit. Thread (Thread) Thread->Exit. Status = Exit. Status; Ke. Query. System. Time (&Thread->Exit. Time) if (! Last. Thread) <Rundown pending APCs> Ke. Terminate. Thread () © Microsoft Corporation 51

Psp. Exit. Thread(status) cont. 2 Process->Exit. Time = Thread->Exit. Time Psp. Exit. Process (TRUE, Process) Process. Token = Ps. Reference. Primary. Token (Process) Se. Audit. Process. Exit (Process); Ps. Dereference. Primary. Token. Ex (Process, Process. Token) Ob. Kill. Process (Process) // Rundown the handle table Ob. Dereference. Object (Process->Section. Object) Psp. Exit. Process. From. Job (Process->Job, Process) <Rundown pending APCs> Mm. Clean. Process. Address. Space (Process) Ke. Set. Process (&Process->Pcb, 0) // signal the process Ke. Terminate. Thread () © Microsoft Corporation 52

Psp. Exit. Process(Last. Thread, Process) Ob. Dereference. Object (Process->Security. Port) if (Last. Thread) <Notify registered callout routines of process exit> <Finish cleaning up Job Object> return // we were called from Psp. Delete. Process() Mm. Clean. Process. Address. Space (Process) © Microsoft Corporation 53

Ke. Terminate. Thread() <Raise to SYNCH_LEVEL, acquire process lock, set swap busy> <Insert the thread in the reaper list> <Acquire dispatcher lock> <Queue reaper work item if needed> if (Thread->Queue) Remove. Entry. List(&Thread->Queue. List. Entry) Ki. Activate. Waiter. Queue (Queue) Remove. Entry. List(&Thread->Thread. List. Entry) // from parent’s list <Release process lock without dropping IRQL> Thread->State = Terminated Process->Stack. Count -= 1 <Initiate an outswap of the process if Stack. Count now 0> Ki. Rundown. Thread (Thread) // rundown arch-specific data <Acquire dispatcher lock> Ki. Swap. Thread (Thread, Current. Prcb) // yield processor final time © Microsoft Corporation 54

Psp. Process. Delete () <Remove the process from the global list> Psp. Remove. Process. From. Job (Process->Job, Process) Ob. Dereference. Object. Defer. Delete (Process->Job) Ob. Dereference. Object (Process->Debug. Port) Ob. Dereference. Object (Process->Exception. Port) Ob. Dereference. Object (Process->Section. Object) Psp. Delete. Ldt (Process) Ke. Stack. Attach. Process (&Process->Pcb, &Apc. State) Ob. Kill. Process (Process) Psp. Exit. Process (FALSE, Process) Ke. Unstack. Detach. Process (&Apc. State) Mm. Delete. Process. Address. Space (Process) Ex. Destroy. Handle (Psp. Cid. Table, Process->Unique. Process. Id) Psp. Delete. Process. Security (Process) Ob. Dereference. Device. Map (Process) Psp. Dereference. Quota (Process) © Microsoft Corporation 55

Psp. Thread. Delete() Mm. Delete. Kernel. Stack() Ex. Destroy. Handle (Psp. Cid. Table, Thread->Cid. Unique. Thread) Psp. Delete. Thread. Security (Thread) if (! Thread->Process) return // never inserted in process Psp. Lock. Process. Exclusive (Process, Current. Thread) Remove. Entry. List (&Thread->Thread. List. Entry) Psp. Unlock. Process. Exclusive (Process, Current. Thread) Ob. Dereference. Object (Process) © Microsoft Corporation 56

Summary: Native NT Process APIs Nt. Create. Process() Nt. Terminate. Process() Nt. Query. Information. Process() Nt. Set. Information. Process() Nt. Get. Next. Thread() Nt. Suspend. Process() Nt. Resume. Process() Nt. Create. Thread() Nt. Terminate. Thread() Nt. Suspend. Thread() Nt. Resume. Thread() Nt. Get. Context. Thread() Nt. Set. Context. Thread() Nt. Query. Information. Thread() Nt. Set. Information. Thread() Nt. Alert. Thread() Nt. Queue. Apc. Thread() © Microsoft Corporation 57

Discussion © Microsoft Corporation 58