Module 3 Transaction Processing Monitors COP 6730 1

Module 3 Transaction Processing Monitors COP 6730 1

Transaction Processing Monitor • Transactional RPC. • Functional Principles of the TP Monitor. • Managing Request and Response Queues. • Other tasks of TP Monitors – Load Balancing – Authentication and Authorization – Restart Processing 2

Layering of Services in a Transaction Processing System • Basic Operating System (BOS) • Transaction Processing Services (TRAPS) TP Monitor • Transaction Processing Operating System (TPOS) 3

Basic Operating Systems • The Basic Operating System (BOS) uses the hardware directly. • It knows little or nothing about transactions • Services include: Process/Thread scheduling, file system, simple sessions, authentication, etc. 4

Transaction Processing Operating System (TPOS) • TPOS renders the BOS objects and services in a transactional style. • Example Services: – Processes are bound to transactions – Messages must not be delivered until the transaction has committed 5

Transaction Processing Services (TRAPS) • TRAPS uses both BOS and TPOS to create a transaction-oriented programming environment. Services include: load control, resource managers, database, etc. • The application typically uses the TRAPS and TPOS, depending on the application's sophistication. Note: These layers do not reflect a strict implementation hierarchy. Rather, it describes a separation of concerns among different system components (e. g. , TPOS can implement some of its data structure using SQL). 6

Transaction Processing (TP) Monitor 7

Transactional Remote Procedure Calls (1) Transaction control must be exercised over all resource manager interactions within one transaction (a “web” of the transaction’s calls): Control of participants: § The Transaction Manager (TM) must track which resource managers have been called. § At commit time, the TM must go Yes out and ask each resource manager involved whether they agree to successfully terminate the transaction. Ready ? Yes 8

Transactional Remote Procedure Calls (2) – Preserving transaction-related information: Resource Manager (Server class) VOTE Server Resource Manager (Server class) VOTE § The same resource manager can be invoked more than once by a transaction ØAt commit each process running the same resource manager code is asked individually about its commit decision. § Sometime, the resource manager can only vote on commit when information pertaining to that transaction is in one place. ØIf the different invocations left their traces in different servers, the TRPC mechanism must provide a means to relate multiple invocations of the same resource manager to each other. 9

Transactional Remote Procedure Calls (3) Support of the transaction protocol: § The resource managers must stick to the rules imposed by the ACID paradigm. Resource Manager Regular code Other Resource managers Code for participating in transaction model Transaction Manager 10

CALLBACK • Each resource manager has a service interface that defines what the resource manager does for living. • Furthermore, it declares a number of callback entries, which can be called when certain events happen. Resource Manager Service interface Other Resource managers Regular code Code for participating in transaction model Callback entries Transaction Manager 11

CALLBACK • Each resource manager has a service interface that defines what the resource manager does for living. • Furthermore, it declares a number of callback entries, which can be called when certain events happen. – If the RM maintains any durable objects, it must be able to accept rollback and prepare/commit callbacks from the TM. – The RM exports the callback entries to the transaction manager when it first checks in with the TP monitor. – The number of callback entries provided by a RM depends on its level of sophistication. Note: This can result in multiple threads executing the RM’s code at different entry points. 12

• Boolean rm_Prepare(); /* invoked at phase 1 of commit. Returns vote. */ • Boolean rm_Rollback_Savepoint (Savepoint); /* rollback to requested savepoint number */ • Boolean rm_Commit (Boolean); /* invoked at phase 2 commit; param is decision */ • Boolean rm_Savepoint (); /* invoked when the transaction takes a savepoint */ • (void) rm_UNDO (&buffer); /* asks the resource mgr. to undo the effects */ /* described in the log record passed as parameter */ • (void) rm_Abort (); /* invoked at end of abort phase */ /* after all undo steps have been performed */ • (void) rm_REDO (&buffer); /* asks the resource mgr. to redo the effects */ /* described in the log rec. passed as a param */ • (void) rm_Checkpoint (); /* invoked by TM for taking a check point */ • (void) rm_restart (LSN); /* first callback from the transaction manager at */ /* restart; passes the address of the resource */ /* manager’s recent checkpoint record */ The invocation of a resource manager by the transaction manager at such callback entry is just another transactional remote procedure call. CALLBACK ENTRIES 13

Resource Manager’s Interfaces Three groups of interfaces: 1. The interface the resource manager exports, 2. the interfaces it uses other general resource managers, and 3. the system resource manager interfaces it uses to support the transaction protocol. 14

Install A New Resource Manager (1) RMID rm. Install (RMNAME, &rm_callbacks, Access. Control. List, /* authorize requests*/ stuff); /*other info. For TP monitor*/ • RMNAME is the user-supplied global external ID. • &rm_callbacks is an array of pointers to the callback entries. – These callback entries are specified as relative addresses in the load module for that RM. – The TP monitor keeps this array in the TRPC stub of the respective RM’s address space. 15

Install A New Resource Manager (2) RMID rm. Install (RMNAME, &rm_callbacks, Access. Control. List, /* authorize requests*/ stuff); /*other info. For TP monitor*/ • Access. Control. List allows the TP monitor to check whether an incoming request for that RM is acceptable from a security point of view. • Stuff denotes remaining parameters: – The location from which the code for the resource manager can be loaded. – Which other resource managers need to be available before this one can start, etc. 16

Complex Interaction If a client C repeatedly invokes server class C, how should that be handle? We need to consider three scenarios. Give me the next record 17

Complex Interaction If a client C repeatedly invokes server class C, how should that be handle? We need to consider three scenarios. Give me the next record Mail must have header, body, and trailer 18

Case A: Independent Invocations • A server class S can be called arbitrarily often, and the outcome of each call is independent of whether or not the server class has been called before. Example: - The server reads records from a database. - It performs some statistical computations on them, and returns the result. • Since the server keeps no state about the call, it can declares its consent to the transaction’s commit upon return. 19

Case B: Invocation Sequence • The client wants to issue requests that explicitly refer to earlier service requests. Example: The mini-batch example updates a number of records at a time (in a subtransaction) using a cursor. • The final decision of the server class depends on the outcome of the last invocation. I can decide for this server class 20

Case C: Complex Interaction • Each call to S is independent, except for some global integrity constraints that cannot be checked until commit and that may depend on the results of all previous invocations. • The server class must remember the results of all invocations by the client until commit. 21

Complex Interaction – Example – A mail server stores the various parts of a message in a database. – It maintains a consistency constraint that says that all mails must have at least a header, a body, and a trailer. There must be some way of relating all the updates done by the same client when the server is called (back) at rm_Prepare. 22

Implementing state associated with a client-server interaction Context information about multiple invocations of the same server class can be maintained in four different ways: 1. Context is maintained by a communication session. 2. Context is passed back and forth. 3. The servers write context information to a database. 4. The context is stored in a segment of shared memory for all servers of the same class. 23

Context Management: Communication Session Subsequence requests go to the same server Client Context In local variables Session Request/ Response Server S 1 Context In local variables • Such a session has to be managed by the TPOS. • All the client or the server has to do is to declare that it needs one. – Typically, the request for a session is issued by the server. Applications do not want to be bothered with TPOS. • The price for this is considerable overhead in the TPOS for maintaining and recovering sessions. 24

Session Management • If context maintenance is handled thru communication session, then the TP monitor is responsible for binding a server process to one client. • A Bind. ID uniquely identifies an association between a client instance and a server instance. typedef struct{ rm. Instance Uint End. A; End. B; Seq. No; }Bind. Id; /* /* one end of stateful interaction /* other end of stateful interaction /* sequence no. to distinguish between multiple stateful interactions between the same servers. /* handle for a stateful interactions */ */ */ Bind. ID rm. Bind (rm. Instance); /* function passes the ID of the client to which /* a session with the server has to be established. /* gets back a handle which identifies the interaction /* between this server and this client. /* returns the Null instance if binding fails. */ */ */ Boolean rm. Unbind (Bind. ID) /* inverse to rm. Bind, waives the future use of the /* specified binding, returns FALSE if no binding /* with that identifier was in effect */ */ */ 25

Context Mgmt. Technique: Context Passing (1) • Context is passed back and forth explicitly between client and server upon each request and reply. This approach relieves the TPOS of the problem of context management. 26

Context Mgmt. Technique: Context Passing (2) Context I do not have “context” ? Server S 3 Prepare Transaction Manager Since rm_Prepare call from the TM carries no context in its parameter list, the commit may have to be done as follows: – The client issue a “final” service call (with the “final” context), which tells the server that the client is about to call commit. – When the server returns normally, the client can call Commit_Work, and the server will not be called back by the Transaction Manager 27

Context Mgmt. Technique: Keep context in Database (1) • The servers keep the context in a database. An arbitrary instance of the server can be invoked to vote on commit. • The state information must be supplied with the key attributes: – TRID – Client RMID – Sequence number of the invocation in order to uniquely identify which thread of control it belongs to (see next slide). 28

Key Attributes to Access Context Information Server Class A resource manager on i t c nsa a r T 1 A 1 T Trans. T, Client A 1, 2 nd call 2 Transaction B A 2 B 1 B 2 29

Context Mgmt. Technique: Keep context in Database (2) SQL DBMS • Some of the work involved in maintaining the context can be off loaded to either an SQL database system or to the context management service of the TP monitor. 30

Context Mgmt. Technique: Shared Memory (1) • All servers of the same class share a segment of read/write memory in which invocation context is kept. • Synchronization on the shared memory is done by the server. 31

Context Mgmt. Technique: Shared Memory (2) • This scheme is similar to the solution using a context database, except that now the whole responsibility is with the server (class). • This solution is used by very sophisticated resource managers, such as SQL database system. 32

Client-Oriented Context • Client-Oriented Context reflects the state of an interaction between a client and a server. • In the last few pages, we assumed this type of context. 33

Transaction-Oriented Context This type of context is bound to the So. C established by a transaction rather than to an isolated clientserver interaction. time t 1 Server Class S 1 time t 2 Server Class C Server Class S 3 time t 3 Server Class S 2 time t 4 (needs the context established by the earlier call to S 3 from S 1) Note: The context needed by S 3 is not bound to any of the previous interactions with S 2, but it is bound to the transaction as such. 34

General Context Management • A general context management scheme must be able to cope with both type of state information. • The ground rule: • Each server that manages persistent objects must be implemented such that it can keep transaction-oriented context. • The TRPC mechanism must provide means for servers of any type to establish a stateful interaction with a client in case this should be needed (client-orient context). Note: Communication sessions can only be used to support client-oriented context. (We will discuss transaction-oriented management techniques in more details later. ) 35

Managing Resource Managers • The TP monitor’s primary function is to handle TRPCs This is what a typical TP monitor will put atop a conventional O. S. • TPOS maintains a core data structures to manage the resource managers and the resources pertaining to them 36

Anchor of TPOS_Anchor is one well-defined point in TPOS, from which all the system data structures can be reached. typedef struct anchor * TPOS_Anchor. P; typedef struct anchor /* TPOS anchor for the global control blocks */ {. . . TPAnchor. P TPMon. CBs; /* pointer to the anchor of the TP monitor */ TMAnchor. P TM_CBs; /* pointer to the anchor of transaction mgr. */ CMAnchor. P SM_CBs; /* pointer to the anchor of session (comm. ) mgr. */ LMAnchor. P LM_CBs; /* pointer to the anchor of log mgr. IMAnchor. P IM_CBs; /* pointer to the anchor of isolation (lock) mgr. */ */ } TPOS_Anchor • Each system RM (e. g. , transaction manager) has its own local anchor, where this RM’s data structures are rooted. • The declaration shows five system RMs. This is a basic set; real system might have more. 37

Anchor of TPOS - Rationale • TPOS comes up before any of the RMs is activated upon system startup • Keeping the pointer to the anchors of system RMs in TPOS_Anchor ensures that addressability can be established in an orderly manner upon system startup. TPOS _Anchor System RM RM 38

TP Monitor’s Anchors • The anchor of a system resource manager has no fixed layout. • The TP monitor’s anchor might have the following structure: typedef { struct tpa * TPAnchor. P; struct tpa char handle Description RMID of resources RMCB managed by PCB TP Monitor SECB } TPAnchor; /* TP monitor anchor for its control blocks */ *My. Version’ /* version of the TP monitor running */ Repository[64]; /* name of the repository I work with */ /* repository holds description of RM’s */ Repos_Handle; /* handle for calls to repository */ Contrext. DB; /* name of database for keeping context Ctx. DB_Handle; /* handle for calls to context database */ Next. RMID; /* next RMID to be handed out */ First. RMCB; /* pointer to 1 st res. mgr. control block */ First. PRCB; /* pointer to 1 st process control block */ First. SECB; /* pointer to 1 st session control block */ */ • These system control blocks are implemented as semaphores to make sure that a process can operate on these control data structures without being disturbed by other processes 39

Central Control Blocks • Descriptive data about processes, transactions, and RMs are kept in central control blocks. • There is one system RM that is responsible for and encapsulates each type of control block. Examples (also see next page): - TP monitor is responsible for the resource manager control blocks, the process control block, and the session control blocks (see last page). - The transaction manager is responsible for the transaction control block, and so on. 40

Central Data Structure A system RM A central control blocks There is one system RM that is responsible for and encapsulates each type of control block. 41

Accessing Central Data Structure of TPOS 1. The following function can be called from any process: PID My. Procid (); TRID My. Trid (); /* returns the identifier of the process the caller is running in /* this is actually a call to the basic OS /* returns the transaction identifier the caller is executing within RMID My. RMID (); /* returns the RMID of the RM that has issued the call */ RMID Client. RMID (); /* returns the RMID of the caller’s client */ */ 2. The following function can only be called from system resource manager: PCB My. Proc (); /* returns a copy of the central process control block /* describing the process the caller is running in /* returns copy of the central transaction control block /* describing the transaction the caller is working for /* returns copy of the central resource manager control block /* describing the res. mgr. that issued the call /* like My. RM, but for the caller’s client */ */ Trans. CB My. Trans (); RMCB My. RM (); RMCB Client. RM (); PCB * My. Proc. P (); /* returns a pointer to the central process control block /* describing the process the caller is running in Trans. CB * My. Trans. P (); /* returns a pointer to the central transaction control block /* describing the transaction the caller is working for RMCB * My. RMP (); /* returns a pointer to the central resource mgr. control block /* describing the res. mgr. that issued the call RMCB * Client. RMP (); /* like My. RMP, but for the caller’s client */ */ 42

My. Trans vs My. Trans. P • My. Trans gets a copy of all descriptive data for the current transaction. • My. Trans. P provides, in addition, linkage information to other relevant control blocks. 43

Protection Domains • Subsystems typically want to protect themselves from faults in the application (i. e. , caller. ) • They want to encapsulate their data so that only they can access them directly. • Such an encapsulated environment is called a protection domain. • The system RMs should operate as protection domains 44

Two Ways to Provide Protection Domains 1. Process as protection domain 2. Address space as protection domain 45

Process as Protection Domain – Each process has private address RM 1 space. – Service requests are handled by switching processes; that is, by sending a message to the server process. Note: A domain switch requires a process switch, and it may require copying parameters and results between the processes. Example: UNIX Process 1 Address 1 Service request Copy parameters Address 2 Process 2 RM 2 46

Address Space as Protection Domains § A process has many address spaces, one for each protected callee and one for the caller. § A service request is handle by switching address spaces, i. e. , caller’s process can execute code of the callee § The address space protection domain of the callee contains: – some of the caller’s segments, and – the address space belonging to the callee. 47

Switching Address Space Call RM 2 Primary process of RM 1 Process 1 Address 1 (RM 1) Primary Address 2 (RM 2) Switch Running Switching my own address code space Address 3 (RM 3) This is more efficient than process switch. Example: IBMAS/400, IBM MVS/XA, VAX-VMS. 48

Association of Processes with RMs • Each process belongs to a fixed server class, and has a RMID associated with it (i. e. , ID of the primary RM) • At run time, however, a process can switch to other address spaces (by issuing TRPCs) which run for other RMIDs • This ability of a process to execute in different spaces requires the system calls My. RMID and Client. RMID (example on next page). 49

Association of Processes with RMs Note: A process cannot switch to any address space for both scheduling and protection reasons. 50

Association of Processes with RMs Note: A process cannot switch to any address space for both scheduling and protection reasons. 51

Relationships Described by TPOS’s Central Data Structures 52

Central Data Structures TPMon. CBs TPOS_Anchor TM_CBs SM_CBs LM_CBs IM_CBs TPAnchor TMAnchor First. RMCB First. PRCB First. SECB (sessions) SMAnchor (Comm. Mgr. ) LMAnchor IMAnchor (Log Mgr. ) (Lock Mgr. ) First. TACB PRCB (processes) RMCB TACB (resource mangers) (transactions) PRTA_CB PRRM_CB RMTA_CB RMPR_CB RMNO_CB TARM_CB (suspended transactions) (addr. spaces I may switch to) (transactions currently working for) (processes may Switch to this server class) (where this RM is available) (RMs used by this transaction) 53

Resource Manager Control Block We assume a Linked List of control blocks for the instances of each entity type. (We could have easily declared them as relations. ) /*** control block structure for all resource manager (server classes) at the node */ typedef struct Res. Mgr * RMCBP; /* */ typedef struct Res. Mgr /* control block for describing the known resource */ { /* managers at that node */ RMNAME rmname; /* global name of the resource manager */ RMID rmid; /* the resource manager short name (identifier) */ Boolean RMLocal; /* indicates if the RM available at the local node */ pointer acl; /* pointer to access control list to authorize requests */ /* verify if the client is authorized to perform the operation */ Uint priority /* priority of this RM for local access scheduling */ Boolean RMactive; /* indicates if the RM is activated or deactivated */ Boolean RMup; /* status flag says resource manager is up or down */ Boolean Up. After. REDO /* says if the resource manager can operate normally */ /* after REDO recovery has completed */ Uint Queue. Length; /* number of requests waiting for that server class */ RMQUEP waiters; /* pointer to the request queue for that resource mgr. */ RMQUEP end_of_chain; /* points to last waiting request; append happens here */ /* queue information refers to local server class only */ /* Continue in the next slide */ 54

Resource Manager Control Block (Cont’d) For load balancing RMTA_CBP RMTA_chain; RMPR_CBP RMPR_chain; RMNO_CBP RMNO_chain; procedure rm_Startup; rm_Shutdown; /* points to the 1 st control block describing a /* transaction the RM is currently working for /* points to chain of control blocks for the processes /* allocated to this resource manager /* points to chain of control blocks for the nodes in the /* network where this resource manager is available */ */ */ /* invoked at restart /* invoked at system shutdown */ */ /* more stuff needed by the transaction manager, the log manager, and others RMCBP } RMCB; next_RMCB; /* next in the list of RM control blocks /* */ */ */ We assume a Linked List of control blocks for the instances of each entity type. (We could have easily declared them as relations. ) 55

Resource Manager Control Block (Cont’d) RMTA_CBP RMTA_chain; RMPR_CBP RMPR_chain; RMNO_CBP RMNO_chain; procedure rm_Startup; rm_Shutdown; /* points to the 1 st control block describing a /* transaction the RM is currently working for /* points to chain of control blocks for the processes /* allocated to this resource manager /* points to chain of control blocks for the nodes in the /* network where this resource manager is available */ */ */ /* invoked at restart /* invoked at system shutdown */ */ /* more stuff needed by the transaction manager, the log manager, and others RMCBP } RMCB; next_RMCB; /* next in the list of RM control blocks /* */ */ */ • There is no control block for server classes. • The relationship between server classes and resource managers is 1: 1 • Both entity types can be represented by just one type of control block. 56

Central Data Structures TPMon. CBs TPOS_Anchor TM_CBs SM_CBs LM_CBs IM_CBs TPAnchor TMAnchor First. RMCB First. PRCB First. SECB (sessions) SMAnchor (Comm. Mgr. ) LMAnchor IMAnchor (Log Mgr. ) (Lock Mgr. ) First. TACB PRCB (processes) Just discussed this RMCB TACB (resource mangers) (transactions) PRTA_CB PRRM_CB RMTA_CB RMPR_CB RMNO_CB TARM_CB (suspended transactions) (addr. spaces I may switch to) (transactions currently working for) (processes may Switch to this server class) (where this RM is available) (RMs used by this transaction) 57

Central Data Structures TPMon. CBs TPOS_Anchor TM_CBs SM_CBs LM_CBs IM_CBs TPAnchor TMAnchor First. RMCB First. PRCB First. SECB (sessions) SMAnchor (Comm. Mgr. ) LMAnchor IMAnchor (Log Mgr. ) (Lock Mgr. ) First. TACB PRCB (processes) We now discussed this RMCB TACB (resource mangers) (transactions) PRTA_CB PRRM_CB RMTA_CB RMPR_CB RMNO_CB TARM_CB (suspended transactions) (addr. spaces I may switch to) (transactions currently working for) (processes may Switch to this server class) (where this RM is available) (RMs used by this transaction) 58

Process Control Block */ */ Switch Address 2 (RM 2) Address 1 (RM 1) Instance. Of ***/ */ Address 3 (RM 3) Primary k Pro ces s /** control block structure for all processes in server classes typedef struct Processes *PCBP; /* typedef struct Processes /* control block describing a process in a server class { /* and its dynamic associations PID pid; /* process no. provided by the basic OS Primary RM RMID Instance. Of; /* this is the process’ own server class RMID Runs. In; /* server class of res. mgr. process runs in now RMID Client. ID; /* RMID of client having invoked Runs. In TRID Works. For; /* Trid of transaction under which process currently runs Boolean busy; /* does the process currently service a request ? Uint priority; /* priority the process has currently been assigned PRTA_CBP TAs. To. Do; /* pointer to a list of suspended transactions PRRM_CBP IMay. Use; /* list of res. mgrs. to whose address spaces I may switch */ } PCB; Runs. In 59

Capture m: n Relationship /** control block structure for all processes in server classes typedef struct Processes *PCBP; /* typedef struct Processes /* control block describing a process in a server class { /* and its dynamic associations PID pid; /* process no. provided by the basic OS. . . PRTA_CBP TAs. To. Do; /* pointer to a list of suspended transactions PRRM_CBP IMay. Use; /* list of res. mgrs. to whose address spaces I may switch */ } PCB; typedef struct { RMNAME RMID Res. Mgr * Res. Mgr rmname; rmid; RMTA_chain; RMPR_CBP RMPR_chain; RMCBP } RMCB; next_RMCB; */ Implement m: n relationship between processes & RM’s RMCBP; /* */ /* control block for describing the known resource */ /* managers at that node /* global name of the resource manager /* the resource manager short name (identifier) */ */ */ From page 56 . . . RMTA_CBP ***/ */ */ /* /* points to the 1 st control block describing a */ /* transaction the RM is currently working for */ /* points to chain of control blocks for the processes */ /* allocated to this resource manager */ /* next in the list of RM control blocks */ 60 */

RMPR Chain: Process List 1. This one lists all the processes that may switch to this server class /*** typedef struct { PID Boolean RMPR_CBP } RMPR_CB template for an entry in the resource manager’s process queue ***/ rmpr *RMPR_CBP; /* */ rmpr /* control block describing a process associated with the */ /* resource manager */ pid /* process identifier of the associated process */ Primary. Proc /* is this a process that was allocated for the resource */ /* manager’s server class ? */ Next. Prcoess; /* pointer to next entry in process list */ /* */ Res. Mgr Process Res. Mgr. Name rmid RMPR_Chain Procid Instance. Of Process. ID Primary. Proc Next. Process RMPR Control Blocks IMay. Use 61

PRRM Chain: Resource Mgr. List Process Res. Mgr. Name rmid RMPR_Chain k rmid Next. Res. Mgr PRRM Control Blocks Process Procid Instance. Of IMay. Use Address 1 (RM 1) Primary Address 2 (RM 2) Switch Address 3 (RM 3) 2. This one lists the RMIDs whose code a process is allowed to execute. /*** typedef struct { RMID char PRRM_CBP } PRRM_CB template for an entry in the process’ resource manager list prrm *PRRM_CBP; /* prrm /* control block describing a resource manager the /* process can switch to rmid; /* ID of resource manager implemented by the /* server class * stuff; /* addressing information, depends on the OS Next. Res. Mgr; /* pointer to next entry in resource manager list ***/ */ 62

M: N Relationship between RM’s and Processes PRRM Control Blocks rmid Process Next. Res. Mgr Procid Process Instance. Of Res. Mgr. Name rmid IMay. Use PMPR_Chain Process. ID Primary. Proc Next. Process RMPR Control Blocks 63

Resource Managers & Transactions TPMon. CBs TPAnchor First. RMCB First. PRCB First. SECB TPOS_Anchor TM_CBs SM_CBs TMAnchor SMAnchor (Comm. Mgr. ) LM_CBs IM_CBs LMAnchor IMAnchor (Log Mgr. ) (Lock Mgr. ) First. TACB SECB TACB (sessions) (transactions) TARM_CB (RMs used by this transaction) 64

Control Blocks for the m: n Relationship between RMs and Transactions m: n relationship between Res. Mgr & Transactions 65

RMTA Control Block for 1: n Relationship from RM’s to Transactions Template for a control block containing information about a transaction’s association with a RM; anchored at RMTA_chain. typedef struct rmta *RMTA_CBP; rmta /* control block describing the association between /* one resource manager and one transaction */ */ { TRID pointer Serviced. TA; Data. INeed; RMTA_CBP Next. TA; } RMTA_CB; /* ID of the transaction involved /* points to a data structure the RM may want to maintain /* for keeping state pertaining to its work on that TA /* points to control block for next transaction of RM */ */ • The variable Data. INeed allows a resource manager to maintain context for a transaction (i. e. , transaction-oriented context) • These control blocks are rooted in the RMCB (see Slide 64) • Similarly, we have TARM control blocks for the 1: n relationship from transactions to RM’s (see Slide 64). 66

Waiting Queue Resource Mgr. 67

Resource Manager’s Request Queue /*** template for an entry in the resource manager’s request queue typedef struct { rmq *RMQUEP; pointer /* processes Request. Msg; pointer long RPC_Data timeout RMID rm. Instance RMQUEP } RMQUE; Client. Type; Client. Inst; Next. Waiter; ***/ /* control block describing one TRPC to that res. manager*/ /* that could not be scheduled immediately for lack of */ */ /* pointer to the message contents; depends on the */ /* interface that the request is directed to */ /* points to a control block describing the RPC context */ /* defines a timeout interval after which the waiting */ /* request is cancelled */ /* which resource manager issued the request */ /* which instance of the RM issued the request */ /* pointer to next entry in request queue */ /* */ Note: RPC_Data enables “context passing” 68

Central Data Structures TPMon. CBs TPOS_Anchor TM_CBs SM_CBs LM_CBs IM_CBs TPAnchor TMAnchor First. RMCB First. PRCB First. SECB (sessions) SMAnchor (Comm. Mgr. ) LMAnchor IMAnchor (Log Mgr. ) (Lock Mgr. ) First. TACB PRCB (processes) RMCB TACB (resource mangers) (transactions) PRTA_CB PRRM_CB RMTA_CB RMPR_CB RMNO_CB TARM_CB (suspended transactions) (addr. spaces I may switch to) (transactions currently working for) (processes may Switch to this server class) (where this RM is available) (RMs used by this transaction) 69

Session Control Block /*** control block for all sessions an RM in that node participates in ***/ typedef struct Sessions *SECBP; /* */ struct Sessions /* control block describing one high-level (TP monitor) session */ { char name [BIG]; /* the session name */ Boolean incoming; /* session polarity (incoming or outgoing) */ PID Initiator; /* ID of process that initiated the binding */ NODEID Init. Node; /* node where the initiator resides */ PID Other. End /* ID of the bound side of the session */ NODEID Bound. Node /* node where the bound process runs */ HANDLE handle; /* session handle given out by the communications manger */ TRID Used. By; /* transaction that is currently riding on this session */ char * stuff; /* many other things */ } SECB; /* */ Init. Node Bound. Node name Other. End Incoming Trasaction “ Used. By ” Initiator 70

Session Control Overhead • In practice, it is too expensive to establish and release sessions on a per-transaction basis. • Rather, sessions are maintained between processes in different nodes over a longer period of time. ÞThe communication manager must keep track of which transactions are associated with which sessions. 71

Why do We need SMAnchor ? • Session control blocks are rooted in TPAnchor • A session control block describes the users of the session, i. e. , – the process at the two ends – the transaction riding on this session • The Communication Manager (using SMAnchor) is responsible for recovering sessions after system restart (Slide 74). Session control blocks must be ready before this recovery process. 72

Central Data Structures TPMon. CBs TPOS_Anchor TM_CBs SM_CBs LM_CBs IM_CBs TPAnchor TMAnchor First. RMCB First. PRCB First. SECB (sessions) SMAnchor (Comm. Mgr. ) LMAnchor IMAnchor (Log Mgr. ) (Lock Mgr. ) First. TACB PRCB (processes) RMCB TACB (resource mangers) (transactions) PRTA_CB PRRM_CB RMTA_CB RMPR_CB RMNO_CB TARM_CB (suspended transactions) (addr. spaces I may switch to) (transactions currently working for) (processes may Switch to this server class) (where this RM is available) (RMs used by this transaction) 73

Restart Processing - Overview All sessions are ready Startup Script Bri ngs BOS RM CM p tu tar up s RM TPOS Log records star tup startup RM TM ify t Iden rds Log co e r g Lo O ND U / O ED R 74

Restart Processing 1. After a crash, the BOS performs the following: 1. 1 The BOS first bootstraps itself back into existence. The log must come up early with the BOS. 1. 2 The BOS performs its restart according to a startup script that brings up the TPOS. 2. The TPOS performs the following: 2. 1 Format all the system control blocks needed for managing the RMs. 2. 2 Load the description of all local RMs into RMTable, and enter its processes into PRTable. 2. 3 Execute the TPOS startup script (details in page 75) 2. 4 Bring up all the RMs at that node and orchestrating their recovery (details in page 76) 75

TPOS Startup Script • Prepare the Servers: – For each active RM, start the prespecified number of processes for that server class and load the code. – Update the PRTable. • Determine condition at the crash: – Call the TM at its rm_Startup entry. – The TM reads the log to determine the active transactions at the time of the crash, as well as the RM involved. – The TM formats the TATable accordingly. • Recover transactional sessions: – Call the Communications Manager (CM) at its rm_ Startup entry. – The TM feeds all relevant log records to the CM in order for it to recover its transactional sessions. 76

Recover the RMs • RMs are brought up according to the sequence prescribed in the TPOS startup script. (This is to make sure that services required for the restart of other RMs are in place before they are brought up). • When a RM can be brought up, the TP monitor activates it at the rm_Startup entry point. – The RM then does whatever is needed for its own initialization, and – eventually calls the Identify function of the TM. • The TM scans the log to see if there is any recovery work to do for that RM. If so, i. it feeds the REDO records to the RM thru the rm_REDO callback entry. ii. After that, the same is done for the UNDO records. • The TM returns from the Identify call, and the RM eventually returns from the startup call. 77

Restart a Resource Manager TP Monitor Resource Manger R Transaction Manger rm_Startup Identify rm_REDO rm_UNDO Repeat for all REDO log records Repeat for all UNDO log records Invoke a RM Return from a RM invocation 78

Mid-term Exam • Date: October 7, 2014 • Module 1, Module 2, and Module 3 (up to Slide 78) 79

Applications of Queues Load Control: If there is a temporary peak in the request rate for a RM, the requests can be put into a temporary queue in front of server class (rather than flood the system with new processes). RM RM 80

Another Application of Queues End User Control: It is critical for the real actions at the end points of the system to be in accord with the central RMs. (such queues must be kept in durable storage). Teller Machine “dispense $200” Central RMs Thank you. . $$$ “dispense $200” “subtract $200” 81

Application of Queues (cont’d) Recoverable Data Entry: The key point here is not to lose any input data, even if the system crashes before they get processed. (data queues must be kept in durable storage). High input rate Data Queue Transaction Asynchronous Transaction Processing: The queues maintain durable service requests for the next server class. Client A Client B Clients A and B might be identical Server 1 Server 2 Server 4 Server 3 Output queue of Server 3 or Input queue of Server 4 82

Queued Transaction Processing (QTP) For each request, there are three transactions: 1. The 1 st transaction creates the request and enters it into the server’s input queue. 2. The 2 nd transactions processes the request in the server and enters the result into the client’s response queue 3. The 3 rd transaction takes the response out of the client’s response queue and presents it to the user (or does whatever is required). Transaction 1 Send request Transaction 2 Receive request Server Client Receive response Transaction 3 Send response 83

QTP uses Durable Relationships The QTP model implies durable relationships between a resource manager and its queues • Each request and response is recorded durably in a queue • The history of resource manager interactions makes it possible to determine what happened last between a certain client and its server 84

Queuing System as a RM • Special RM: Queuing system can best be thought of as a special RM that handles request and response queues. Queue RM Send request Receive request Server Client Receive response Send response 85

QTP – Client Behavior 1. The client has to be connected to the queuing system in order to issue requests. 2. It then can switch back and forth between sending requests to servers and receiving responses from them. 3. Finally, it can disconnect from the queue only if there are no outstanding responses. Connected Receive or Re-receive Send Request sent Send Receive Response received Receive Disconnected 86

Queue: Recoverable Session Registering a client with a queue establishes a recoverable session between the client and the queue RM. – This is another example of the stateful interaction between a client and a server – After a crash of either side, the session can be resumed with all state information reestablished as of the last successful transaction that updated the queue. Queue RM Recoverable session Send request Client Receive response Receive request Send response Server 87

Queueing System – Data Types typedef char[BIG] QUNAME; /* globally unique name for the queue */ typedef Uint QUID; /* locally unique identifier for the queue */ typedef TRID RQID; /* request identifier, based on transaction identifier */ typedef char REQUEST[ ]; /* request are passed as character strings */ typedef char RESPONSE[ ]; /* response are also passed as character string */ 88

Queuing Operation - Send returns • TRUE if the request was successfully entered into the queue; • FALSE otherwise. Boolean send ( REQUEST Do. This, RQID rqid, QUID To. Queue, QUID Resp. Queue /* Request as a character string */ /* ID of the request */ /* input queue of the server */ /* output queue of the sever */ ); 89

Queuing Operation - Send returns • TRUE if the request was successfully entered into the queue; • FALSE otherwise Boolean send ( REQUEST RQID QUID Do. This, rqid, To. Queue, Resp. Queue /* Request as a character string */ /* ID of the request */ /* input queue of the server */ /* output queue of the sever */ ); 90

Queuing Operation - Receive receives the next response from the queue specified in Resp. Queue. RESPONSE receive (QUID Resp. Queue, Response Keep. That ); Some client-specific data passed in Keep. That are stored by the queuing system if the response has been taken from the queue successfully. Resp. Queue Success Client Keep. That Response 91

Queuing Operation - re. Receive presents the response of the last receive successfully executed by that client. It is used after restart to find out which was the last piece of work it has successfully executed. RESPONSE re. Receive (QUID Resp. Queue); 92

State of a Session typedef struct Session. Parms *Qstate. Ptr; Session. Parms /* parameters describing the state of the session between a client and the queueing system */ { QUID quid; /* ID of the queue involved */ RQID Last. Send; /* ID of the last request sent by the client */ RQID Last. Rcvd; RESPONSE response; /* ID of the request the last response belongs to */ /* informaition passed by the client with the last successful receive operation -- thru Keep. That */ } QState; 93

State Illustration 8 Last. Send 8 Client Queue RM 6 94

State Illustration 8 Last. Rcvd 8 Client Response of Request 6 Queue RM 6 6 NOTE: “Last. Send” = “Last. Rcvd” no outstanding request 95

Opening and Closing a Session • Connect: If the session had not been closed down explicitly before, the system returns the state of the session (as shown in page 93). If the session is newly established, Qstate contains NULL entries. Qstate connect ( RMID My. RMID, QUID To. Queue); • Disconnect: Client closes the session with the specified queue. Boolean disconnect ( RMID My. RMID, QUID From. Queue); 96

A Restart Example: A Ticket Printer The ticket printer advances a counter that can be read from the client process each time a ticket is printed. 13 12 13 14 13 1. 13 Client Server Database Resp. Queue The client reads the counter value (e. g. “ 13”), and receives the response from the Resp. Queue (passing the counter value in the parameter Keep. That). receive (Resp. Queue, 13); 2. The client commits, and then prints the ticket. Note: This helps the client remember that ticket 13 has been printed 97

A Restart Example: A Ticket Printer 3. Upon restart, the Client gets the state of the session prior to the system crash. Connect (My. RMID, Resp. Queue); Last. Sent = Last. Rcvd and Keep. That = prter counter 13 Client t icke t t n i r P Connec t Session state 13 14 Quid Last. Send Last. Rcvd Keep. That Resp. Queue 98

A Restart Example: A Ticket Printer 3. Upon restart, the Client gets the state of the session prior to the system crash. Connect (My. RMID, Resp. Queue); Last. Sent ≠ Last. Rcvd 13 Ticket has been printed. Response to the Subsequent request has Not arrived Client Connec t Session state 13 14 Quid Last. Send Last. Rcvd Keep. That Resp. Queue 99

A Restart Example: A Ticket Printer (cont’d) 3. Upon restart, it gets the last value passed in Keep. That as part of the state of the queue. Connect (My. RMID, Resp. Queue); case 1: Last. Sent = Last. Rcvd. If value (Keep. That) = printer_counter Then The response had not been processed before the crash, so printing must be done. Else The ticket has already been printed. case 2: Last. Sent Last. Rcvd. - Response processing had been completed before the crash, and a subsequent request had been entered. - The response to that request has not yet arrived, so all the client has to do is to enter the request sent state. 100

Load Balancing (1) • How many processes should be given to a server class ? • When a request arrives at a server class, and all processes are busy handling requests: – should a new process be created for the server class, or – should the request wait for a process to become available ? • If the server class is distributed among several nodes, should the request be sent there rather than kept waiting locally. 101

Load Balancing (2) Physical Machine “A” Client X Service Machine “B” A Client Service B Y How many virtual machines of which types should be maintained? and where ? 102

Local Process Scheduling • The O. S. bases its scheduling decisions on a small set of parameters: – Priority of the process: execution priority – Working set size: how much real memory should be available at any given point in time for the process to run properly – Completion time constraints: the request must be processed by a certain time • The O. S. often treats processes as black boxes. It observes their behavior from the outside • TP monitor knows which server classes it creates a process for. It also keeps statistics about that server class utilization. It can do better load balancing. 103

Local Process Scheduling Call A RM Keep the address space Keep the process Send the request to that process Send the request to a lightly loaded node where the service is available Service available in same addr. space Service available in different addr space An idle process provides this service Service only locally available Create a new process for the server class Server class utilization is high CPU utilization is low Once a request is sent to a different note, the scheduling is now that node’s problem No Queue the request 104

Scheduling: Intra-RM Call Process Address Space Keep the Address space call Switch 105

Scheduling: Address Switch Process Address Space Keep the process call Switch 106

Scheduling: Process Switch This Process Address Space I can do it Service is not Process available Address Space call Switch 107

Scheduling: Not Locally Available Service not locally available Process Address Space I do scheduling Scheduler I can do it Switch Node 1 Process Node 3 108

Scheduling: Multiple Candidate Nodes Node 2 I am heavily loaded Node 3 Node 1 Node 4 I am lightly loaded new server 109

Scheduling: All Serves Busy Process Address Space Server Class Address Space We are all busy call new server Switch Only available here Server class Utilization is high CPU Utilization Is low 110

Scheduling: Every node is buzy Process Address Space Server Class We are all busy Address Space call Switch Queue CPU utilization is high or server class utilization is low 111

Average Response Time Independencies of Local Server Loads Increasing concurrency improves system utilization, and therefore response time Increasing concurrency leads to increasing probability of access conflicts and therefore longer average response time Number of Server Processes (Concurrent Transactions) Load balancing performed by the TP monitor must look at both: Basic OS does not do this 1. the utilization of each server class, and 2. the utilization of the underlying physical resources. 112

THRASHING Cannot Improve CPU Utilization Add processes to increase concurrency This phenomenon, known as thrashing, is caused by making scheduling decisions with insufficient information (e. g. , the number of server processes is not considered in load balancing). 113

Thrashing - Illustration Transactions 1 2 D 1 1 2 D 2 1 3 D 3 1 3 D 4 2 3 Processes Data 114

THRASHING (Cont’d) Similar throughput, but transactions take much longer to finish cy n rre cu g sin a re n co Inc Low CPU utilization & high service time (see page 110) 115

Scheduling Across Node Boundaries The scheduling decision has to be based on the following considerations: Sl: expected local response time based on CPU utilization and server class utilization. Sr: expected response time at the remote server, given the utilization of its resources. Tc: communication overhead based on the utilization of the communication medium. If any remote node can be found such that Sr + Tc < Sl , then the request should be executed at that node. 116

Tracking Performance Statistics The TP monitor has to keep track of the performance figures of remote servers as well as local ones. – Load patterns change gradually and performance figures do not have to be kept current to the millisecond. – Periodic messages provide periodic sample of the utilizations, response time, and so on, of the local resources at the remote nodes 117

A Simple Heuristic As long as the utilization samples show that all nodes are running within the pre-specified load limit: Each node can schedule requests to all nodes that provide that service using a round-robin scheme • Advantages: Simple & load evenly distributed • Potential Problem: Control block death (next slide) 118

Control Block Death • Assume N nodes in the system. Each has Ia processes of server class A and Ib processes for server class B. • According to our scheduling model, each instances of A could send its request to any instance of B. • If this configuration operates for a while, each instance of A eventually will be in section with each instance of B. Þ The number of control blocks growing quadratically will get too large. Note: This is likely to happen in large system. 119

Data Affinity (heavily loaded) (lightly loaded) • Lightly loaded Node 2 cannot help because the data are stored in Node 1 • TP monitor does not understand the parameter fields of the data files in making load balancing decisions 120

Authentication • User authentication: Checking the password when a user wants to establish a session with the system (logon). A userid that has been successfully authenticated is called an authorization ID (authid, for short). • Session authentication: This is similar to user authentication, except that the user is mainly a process running on a different node. Challenge response: The client and the server share a secrete encryption key (next slide). 121

Authentication (Challenge Response) First round: The server challenges the client to decrypt a random number. Second round: The client can also authenticate the server by challenging it to decrypt a second random number. SERVER random number N Encrypt secrete key Compare encrypted number EN decrypted number N Decrypt N same secrete key CLIENT 122

Authorization - File Access Control Operating systems use an access matrix to specify which subject is allowed to do what on which object. subject: userid, user group, program, … object: file, directory, program, … what: create, delete, read, write, link, load, and execute. subject object user. A program. P file. F execute read/write user. A can get to file. F only thru controlled channels (i. e. , program. P ). 123

Authorization - Memory Access Control One can get from a lower authority domain to a higher authority domain only through special instructions or O. S. calls 124

TP Monitor: Authentication & Authorization • TP monitor cannot use O. S. services to perform authentication and authorization Reasons: - The units for which O. S. has access control are large and long-lived: process, sessions, and files - TP monitor, on the other hand, has to do authorization on a per-request basis (not on a session basis). 125

TP Monitor: Authentication & Authorization (Cont’d) The TP monitor, upon each service request issued by a user with authid A, has to check whether: – – – this user is allowed to invoke this service (application) from this terminal (application) at this day in the week (time of the day) on this object with these parameters. 126

Criteria for Access Control Time-Sharing Environment § As user logs on, he/she gets a process (or a number of them), which runs under his/her authid. § Programs that are loaded into this process’s address space execute under the same authid, making it easy to decide on whose behalf a request is issued. 127

Criteria for Access Control Client-Server Environment 1 2 1. User is logged in as myself 3 4 2. He decides to check mails 3. TPOS, running under localadm, handles the TRPC by calling Mail Server running under Mailguy 4. Mail Server uses Database System running under Dbadmin What should be the criteria for access control ? 128

Criteria for Access Control Client-Server Environment Who is Myself ? Having each process along the way take on the user’s authid ? myself is not known at all nodes in a distributed system Running process C under myself would result in denial of access (i. e. , myself has fewer rights on Shared resources 129

Criteria for Access Control Client-Server Environment Collect all the authids along the way and specify access rights for these concatenations This is neither manageable nor even required 130

Request-Based Access Control Client-Server Environment This is a two-level scheme: 1. The TP monitor checks the client’s authority to call the server, and passes the transaction’s authid (i. e. , myself ) along with each request. 2. The RM uses the transaction’s authid to decide whether it can provide the service to the client, given that it works on behalf of that user. Note: Since the Database System knows the transaction’s userid (i. e. myself ), it can properly restrict Mailguy’s (client’s) access rights where necessary. 131

Request-based Access Control Client-Server Environment Since the Database System knows the transaction’s userid (i. e. , myself ), it can properly restrict Mailguy ’s (client’s) access right where necessary 132

They are different approaches 1. Page 130: Having each process along the way take on the user’s authid – The user must be recognized at all nodes not possible ! 2. Page 131: Authorize each invocation step individually and tell the server the userid – The local TP monitors check only the client’s authority to call the server; – Checking the transaction’s user is left to the resource manager – possible ! 133

Request-Based Access Control - Properties 1. Restrictive: Mail server can access all tuples in the mail database. However, it is prevented from giving a user tuples that do not belong to his/her mail box. 2. Permissive: - I want to use the mail system to send a file out of my private database to Jim - The mail system calls upon the database, but Mailguy has no access rights on my file. - Knowing, though, that its actually myself calling (i. e. , checking transaction’s authid), these rights can be granted temporarily. 134

Bind-time Authorization (1) When to do authorization ? Bind-time authorization: Whenever a user invokes a RM, the TP monitor do a lookup. Select * from TP_access_matrix where authid = “myself” and nodeid = “here” and rmid = : rmid and entry_name = ‘Service. Entry’; Problem: This is too expensive ! 135

Bind-time Authorization (2) Less Expensive Approach • When the TP monitor loads the code for the application into a process’s address space, it has to resolve all the external references • If the application supplies names of these services, the TP monitor can check right away if the application program makes any unauthorized calls. 136

Run-time Authorization Bind-time authorization cannot be used if the access condition contains time constraints or restrictions on locations. – A solution: The TP monitor keeps an in-memory copy of authorized requests in a data structure that provides fast access via authid. (Avoiding the expensive database access) 137

When to do Session Authentication ? Should a node perform session authentication every time it receives a service request from a remote node ? v A session is kept between the TPOS’s on either side, they have authenticated each other. v If the client TPOS has also authenticated the user, it can indicate this in the request message, suggesting that the server need not do so again. 138