Models and Languages for Coordination and Orchestration IMT

Models and Languages for Coordination and Orchestration IMT- Institutions Markets Technologies - Alti Studi Lucca WS Technologies III Transactions Roberto Bruni Dipartimento di Informatica Università di Pisa 1

Roberto Bruni @ IMT Lucca 23 March 2005 Contents n n Orchestration and Choreography WSFL Transactions and Compensations BPEL 4 WS Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 2

Roberto Bruni @ IMT Lucca 23 March 2005 Motivation I n A top-ranked global management issue in IT: n n "connecting to customer, suppliers and partners electronically" WS Orchestration / Choreography is about n n n providing an open, standard-based approach for connecting WS together to create high-level business process n n n Institutions Markets Technologies IMT executable, abstract, collaborative, reusable reducing complexity, time-to-market, costs increasing efficiency, scalability, adaptability, accuracy Models and Languages for Coordination and Orchestration 3

Roberto Bruni @ IMT Lucca 23 March 2005 Motivation II n Orchestrated WS that are long-running must also manage n exceptions n how the system will respond to faults n n transactional integrity n n Institutions Markets Technologies IMT (when building business process, nearly 80% of time is spent in exception management!) to reach mutually agreed outcomes ACID-based is not ok for long-running transactions compensating transactions are more appropriate Not dealt with in WSFL! Models and Languages for Coordination and Orchestration 4

Roberto Bruni @ IMT Lucca 23 March 2005 Transactions and All the Rest n We shall survey n n Institutions Markets Technologies IMT classic aspects of transactions commitment protocols (postponed after Easter) compensations WS challenges Models and Languages for Coordination and Orchestration 5

Roberto Bruni @ IMT Lucca 23 March 2005 n Concurrency Control and Recovery in Data. Base Systems Concurrency control n n Recovery n n Institutions Markets Technologies IMT is the activity of ensuring that Sw and Hw failures do not corrupt persistent data Transaction n n is the activity of coordinating the actions of processes that operate in parallel, access shared data, and therefore potentially interfere with each other is a sequence of actions on a shared database The goal is to ensure that transactions execute atomically (no interference) Models and Languages for Coordination and Orchestration 6

Roberto Bruni @ IMT Lucca 23 March 2005 Data. Base Systems n State n n Main operations n n n Set of named data items with values Read(x) – query – returns the value of the data item x Write(x, v) – update – sets the data item x to value v We abstract from the granularity of data items n n Institutions Markets Technologies IMT Words in main memory Pages of a disk Records in a file Fields of a record Models and Languages for Coordination and Orchestration 7

Roberto Bruni @ IMT Lucca 23 March 2005 Execution n Operations should be executed sequentially, but some operations can be executed concurrently n The final effect must be the same of some sequential execution 1. Read(x) 2. Write(x, 1) | Read(y) 3. Write(y, 0) Institutions Markets Technologies IMT The order is inessential Models and Languages for Coordination and Orchestration 8

Roberto Bruni @ IMT Lucca 23 March 2005 Transaction Primitives n Start n n n Commit n n n begin a new transaction assign a unique transaction identifier successfully terminate the transaction its effects are made permanent Abort n n n Institutions Markets Technologies IMT invoke the abnormal termination of the transaction its effects are obliterated can be imposed by failures Models and Languages for Coordination and Orchestration 9

Roberto Bruni @ IMT Lucca 23 March 2005 Terminology n Unique entry / exit points n n Committed transactions Aborted transactions Active transactions n n a transaction is defined by a Start operation, followed by a possibly concurrent execution of (Read / Write) operations followed by either a Commit or Abort started but not yet committed (or aborted) Uncommitted transactions n Institutions Markets Technologies IMT active or aborted Models and Languages for Coordination and Orchestration 10

Roberto Bruni @ IMT Lucca 23 March 2005 Database Transactions n n Sequences of operations / actions which preserve database consistency ACID properties: Transaction’s changes to the n n Institutions Markets Technologies IMT Atomicity Consistency Isolation Durability states are ATOMIC: • either all happen • or none happen Models and Languages for Coordination and Orchestration 11

Roberto Bruni @ IMT Lucca 23 March 2005 Database Transactions n n Sequences of operations / actions which preserve database consistency ACID properties: A transaction is a correct n n Institutions Markets Technologies IMT Atomicity Consistency Isolation Durability transformation of the state: its actions, taken as a group, do not violate any integrity constraint associated with the states Models and Languages for Coordination and Orchestration 12

Roberto Bruni @ IMT Lucca 23 March 2005 Database Transactions n n Sequences of operations / actions which preserve database consistency ACID properties: Transactions can be executed n n Institutions Markets Technologies IMT Atomicity Consistency Isolation Durability concurrently, but it appears to each transaction T that others are executed either before T or after T Models and Languages for Coordination and Orchestration 13

Roberto Bruni @ IMT Lucca 23 March 2005 Database Transactions n n Sequences of operations / actions which preserve database consistency ACID properties: Once a transaction successfully n n Institutions Markets Technologies IMT Atomicity Consistency Isolation Durability commits, its changes to the states survive failures Models and Languages for Coordination and Orchestration 14

Roberto Bruni @ IMT Lucca 23 March 2005 Database Transactions n n Sequences of operations / actions which preserve database consistency ACID properties: n n n Atomicity Consistency Isolation Durability Interest on serializability Institutions Markets Technologies IMT n any T 1|…|Tn can be seen as Ti 1; …; Tin Models and Languages for Coordination and Orchestration 15

Roberto Bruni @ IMT Lucca 23 March 2005 Recoverability n When T aborts, its effects are wiped out n Effects on data (updates) n undo n n Effects on each T’ which read values updated by T (dependencies) n cascading aborts n n n Institutions Markets Technologies IMT restore the original value as if T never executed each T’ must be aborted but remember that committed transactions cannot be aborted delaying commits can be necessary to ensure recoverability Models and Languages for Coordination and Orchestration 16

Roberto Bruni @ IMT Lucca 23 March 2005 Recoverability: Example T 1: Start 1 … Write 1(x, 2) … Abort 1 T 2: Start 2 … Read 2(x) … Write 2(y, 3) … Commit 2 1. When T 1 aborts, Write 1(x, 2) must be undone Institutions Markets Technologies IMT T 1|T 2: … Write 1(x, 2) … Read 2(x) … Write 2(y, 3) … Commit 2 … Abort 1 Models and Languages for Coordination and Orchestration 2. Undoing Write 1(x, 2) can compromise T 2 that read from x 3. But T 2 did already commit… it should have been delayed! 17

Roberto Bruni @ IMT Lucca 23 March 2005 Avoiding Cascading Aborts n Cascading aborts are unpleasant, but often necessary to ensure recoverability n n n Require bookkeeping of dependencies Uncontrollably many transactions might be forced to abort Not necessary if every transaction reads values written by committed transactions n Institutions Markets Technologies IMT Delay Readi(x) until all transactions that issued Writek(x, v) are either aborted or committed Models and Languages for Coordination and Orchestration 18

Roberto Bruni @ IMT Lucca 23 March 2005 Strict Execution n When undoing Write(x, v) n which value should be restored? n n n Before image n n n the initial value? but other transactions can have issued Write(x, v’) the value of x just before the operation to be undone Later overwritten values should not be updated Delay Writei(x, v) until all transactions that issued Writek(x, w) are either aborted or committed Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 19

Roberto Bruni @ IMT Lucca 23 March 2005 Why Strict Execution: Example … Write 1(x, 1) Write 1(y, 3) Write 2(y, 1) Commit 1 Read 2(x) Abort 2 Institutions Markets Technologies IMT … Write 1(x, 2) Write 2(x, 3) Abort 1 When T 2 aborts, the value of y should be 3 (the before image) Suppose initially x=1. When T 1 aborts, the value of x should be 3 not 1 Models and Languages for Coordination and Orchestration … Write 1(x, 2) Write 2(x, 3) Abort 1 Abort 2 The before image of T 2 is not meaningful after T 1 aborts 20

Roberto Bruni @ IMT Lucca 23 March 2005 Non-Recoverable Interactions n It is not always possible to reverse effects n n e. g. input / output statements Deferred output approach n n output messages of T only after T commits not always feasible n Institutions Markets Technologies IMT (user input might depend on output messages) Models and Languages for Coordination and Orchestration 21

Roberto Bruni @ IMT Lucca 23 March 2005 More on Serializability n When two transactions execute concurrently, their operations are interleaved n Even if they commit, interference can cause incorrect results n n n Avoid interleaving at all n n Lost updates Inconsistent retrieval correct but inefficient Restrict to serializable executions Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 22

Roberto Bruni @ IMT Lucca 23 March 2005 Lost Updates: Example T_Deposit(a, n): Start t: =Read(a) t: =t+n Write(a, t) Commit Institutions Markets Technologies IMT T_Deposit(a, 10) | T_Deposit(a, 20) : Start 1 Read 1(a) // returns v Start 2 Read 2(a) // returns the same v Write 1(a, v+10) At the end the Write 2(a, v+20) account is increased Commit 2 by 20 (not by 30 as in Commit 1 any serial execution) Models and Languages for Coordination and Orchestration 23

Roberto Bruni @ IMT Lucca 23 March 2005 Inconsistent Retrieval: Example T_Transfer(£, $, n): T_Total(£, $): Start s: =Read(£) s: =s-n t: =Read($) Write(£, s) t: =t+s*rate[£, $] t: =Read($) output(t) t: =t+n*rate[£, $] Commit Write($, t) Commit The output does not correspond to the total amount Institutions Markets Technologies IMT T_Transfer(a, b, 5)|T_Total(a, b): Start 1 Read 1(a) // returns v Start 2 Read 2(a) // returns the same v Write 1(a, v-5) Read 1(b) // returns w Write 1(b, w+5*r) Read 2(b) // returns w+5*r Commit 2 Commit 1 Models and Languages for Coordination and Orchestration 24

Roberto Bruni @ IMT Lucca 23 March 2005 Data. Base System Model I n Transaction Manager (TM) n n n Scheduler n n controls the relative order of execution Recovery Manager (RM) n n receives requests for transactions and operations forwards them to the scheduler responsible for commits and aborts Distributed Architecture n Institutions Markets Technologies IMT sites connected by a network Models and Languages for Coordination and Orchestration 25

Roberto Bruni @ IMT Lucca 23 March 2005 Data. Base System Model II Network Institutions Markets Technologies IMT TM TM TM Sched RM RM DB DB Models and Languages for Coordination and Orchestration TM … Sched 26

Roberto Bruni @ IMT Lucca 23 March 2005 Some ACID Variations I n Flat Transactions n n Traditional model, strictly meets ACID properties Not adequate to model complex situations n n A task’s abort causes all activities to be rolled back Flat with Savepoints n Introduces programmable savepoints n n n Chained transactions n Institutions Markets Technologies IMT Rollback command restore savepoints without aborting Transactions can be rolled back individually Add savepoints via the chain operation n Commits and start a new transaction with same locks Models and Languages for Coordination and Orchestration 27

Roberto Bruni @ IMT Lucca 23 March 2005 Some ACID Variations II n Nested Transactions n Hierarchical tree of transactions n n Top-level is ACID Other nodes are ACI (subtransactions) n n n Institutions Markets Technologies IMT If a parent aborts all its children abort (even if committed) If a subtransaction aborts its parent can still commit A subtransaction commit is seen outside only when its parent commits (protected nested transactions) The structure can exploit parallelism within transactions Localize failures to subtransactions Models and Languages for Coordination and Orchestration 28

Roberto Bruni @ IMT Lucca 23 March 2005 Some ACID Variations III n Multi-level Transactions n Nested transaction that allow n n n pre-commits to export partial results to other transactions at the same level perfect compensations to abort after pre-commit Abstraction layers n Operations at level n are implemented using operations of the lower level immediately below n Institutions Markets Technologies IMT no shortcuts Models and Languages for Coordination and Orchestration 29

Roberto Bruni @ IMT Lucca 23 March 2005 Some ACID Variations IV n Distributed Transactions n n Flat but executed in a distributed environment Partitioned in sub-transactions on network’s nodes n n n Decomposition reflects neither the hierarchical structure of the application, nor its functional structure Decomposition depends on data distribution over the nodes Decisions made by a sub-transaction affect the entire transaction Commit protocols are needed All these models are traditional ones n Institutions Markets Technologies IMT see e. g. [Gray, Reuter 1993] Models and Languages for Coordination and Orchestration 30

Roberto Bruni @ IMT Lucca 23 March 2005 Some Non ACID Flavors I n Long-Lived Transactions (LLT) n Execution takes a considerable amount of time n n n hours, days, weeks, … Not possible to block all short transactions Open Nested Transactions [Gray 1981] n n Compensations are not perfect Relaxed multi-level n n Institutions Markets Technologies IMT No abstraction layers Shortcuts are allowed Models and Languages for Coordination and Orchestration 31

Roberto Bruni @ IMT Lucca 23 March 2005 Some Non ACID Flavors II n Sagas [Garcia-Molina, Salem 1987] n Special case of open-nested n n Only two levels of nesting LLT broken in a non-atomic sequence of ACID sub-transactions T 1…Tn n n Each Ti has a (non-perfect) compensation Ci Partial execution of the sequence are undesirable n n n Institutions Markets Technologies IMT If it cannot be completed, then must be compensated Executed as either T 1…Tn or T 1…Tk. Ck…C 1 Partial effects are visible to other transactions n no notification / compensation to siblings Models and Languages for Coordination and Orchestration 32

Roberto Bruni @ IMT Lucca 23 March 2005 Some Non ACID Flavors III n Flexible [Elmagarmid et al. 1990] n n Same task implemented by different (functionally equivalent) sub-transactions Programmable failure / success dependencies n n n Pre-committed actions and compensations n Institutions Markets Technologies IMT n T 1: Rent a car from AVIZ T 2: Rent a car from HERTS if T 1 fail Mixed (with or without compensations) transactions coexist within a single global transaction Incorporates time for flexible scheduling Models and Languages for Coordination and Orchestration 33

Roberto Bruni @ IMT Lucca 23 March 2005 Some Non ACID Flavors IV n Split and Join Transactions [Pu et al. 1988] n Open-ended activities n Long-running, not foreseeable concurrent interactions n n n Split command n n n IMT Divide a running transaction in two independent ones Can export results, release/share resources Join command n Institutions Markets Technologies Computer-Aided Design/Computer-Aided Manufacturing Editing and Sw development Merges two running transactions into one All resources become shared Results are visible only after the commit of the joint transaction Models and Languages for Coordination and Orchestration 34

Roberto Bruni @ IMT Lucca 23 March 2005 Some Non ACID Flavors V n Multi-Transactions n n To be used in distributed objects systems with heterogeneous databases Complex transactions by composing building blocks (as needed by applications) Mixture of flat, nested, open-nested are possible Countermeasure, Retriable, Timed, … Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 35

Roberto Bruni @ IMT Lucca 23 March 2005 Transactions in GC? n n n n n ACID would generate too many aborts Local consistency vs global consistency Asynchronous commit Heterogeneous policies Several entry / exit points Messages in place of shared data Dynamic topology of the network Mobility For an overview: n "A framework for analyzing mobile transaction models" n Institutions Markets Technologies IMT n A. Coratella, M. Felder, R. Hirsch, E. Rodriguez (Journal of Data. Base Management 12(13), 2001) Models and Languages for Coordination and Orchestration 36

Roberto Bruni @ IMT Lucca 23 March 2005 Some References n Concurrency control and recovery in database systems (Addison-Wesley 1987) n n Transaction processing: concepts and techniques (Morgan Kaufmann 1993) n n IMT J. Gray, A. Reuter Sagas (Proc. SIGMod’ 87, ACM, pp. 249 -259) n Institutions Markets Technologies P. Bernstein, N. Goodman, V. Hadzilacos H. Garcia-Molina, K. Salem Models and Languages for Coordination and Orchestration 37