Models and Languages for Coordination and Orchestration IMT

Models and Languages for Coordination and Orchestration IMT- Institutions Markets Technologies - Alti Studi Lucca Formal Languages for Flow Composition and Compensation: c. CSP Roberto Bruni Dipartimento di Informatica Università di Pisa 1

Roberto Bruni @ IMT Lucca 23 March 2005 Motivation n Key issues in business processes languages for WS n n A plethora of ad hoc proposals for standard n n WS Composition, Orchestration, Choreography WS Transactions Interesting for both Academy and Industry poorly disciplined provision of "accessory" features Design of business processes calls for n n Institutions Markets Technologies IMT rigorous mathematical foundations clean, elegant semantics well-disciplined service composition principles modular implementation 2

Roberto Bruni @ IMT Lucca 23 March 2005 n Flow Diagrams meet Process Description Languages Many proposals to describe business processes unambiguously n XML-based n n Extensions of known mobile calculi n n WSFL, XLANG, WSCI, BPEL 4 WS committed Join, t-calculus, web -calculus Flow-based n Compensating CSP (Butler, Hoare, Ferreira) n Institutions Markets Technologies IMT n previously Structured Activity Compensation (St. AC) Sagas Calculus (Bruni, Melgratti, Montanari) Models and Languages for Coordination and Orchestration 3

Roberto Bruni @ IMT Lucca 23 March 2005 n A L-RT is n n an interactive component of a distributed system which must be executed as if it were a single atomic action In principle (high-level abstraction) n n n Long-Running Transactions (L-RT) it should not be interrupted or fail in the middle and it must not be interleaved with other atomic actions of other concurrently executing components of the system In practice (implementors viewpoint) n it is not easy to keep the illusion alive in (mobile, concurrent) distributed interactive systems! n n n Institutions Markets Technologies IMT external interactions may require undo of committed activities because the transaction is interactive, familiar automatic techniques of check-pointing and rollback are no longer adequate / applicable the illusion of atomicity for a LRT is achieved with the aid of compensation actions supplied by the programmer Models and Languages for Coordination and Orchestration 4

Roberto Bruni @ IMT Lucca 23 March 2005 Compensations n In long lasting negotiations partial agreements can be reached and locally committed by parties n n n to be compensated in case of failure to be published / confirmed on success Formal models are needed n n Institutions Markets Technologies IMT to to discover specification bugs reason rigorously run simulations ease verification Models and Languages for Coordination and Orchestration 5

Roberto Bruni @ IMT Lucca 23 March 2005 n To Compensate n n n Compensations (Merriam-Webster On. Line) to neutralize the effect of to supply an equivalent to to provide with means of counteracting variation to offset an error, defect, or undesired effect The most important fact: n Institutions Markets Technologies IMT Compensations have a cost! Models and Languages for Coordination and Orchestration 6

Roberto Bruni @ IMT Lucca 23 March 2005 Compensations: Examples I n n A client buys books in an on-line bookstore the bookstore debits client's account as the payment for the book order the bookstore later realizes that one of the book in the client's order is out of print to compensate the client for this problem, n n Institutions Markets Technologies IMT the bookstore can credit the account with the amount wrongfully debited and send a letter apologising for their mistake Models and Languages for Coordination and Orchestration 7

Roberto Bruni @ IMT Lucca 23 March 2005 Compensations: Examples II n n Late cancelling of hotel reservations can involve payment of fees Failures on credit checks can cause the abort of parallel activities (which can be partially completed) e. g. n n n to unpackage the goods to be sent to cancel the courier booking Statements of politicians typically require an unbounded number of nested compensations Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 8

Roberto Bruni @ IMT Lucca 23 March 2005 Sagas n Compensation is important when a system cannot control everything, such as when interaction with other agents (including humans) is involved n Garcia-Molina and Salem exploit the concept of compensation to define the notion of a saga n n Institutions Markets Technologies IMT a saga partitions a L-RT into a sequence of several smaller subtransactions, where each of the sub-transactions has an associated compensation If one of the sub-transactions in the sequence aborts, the compensation associated with those committed subtransactions is executed in the reverse order Models and Languages for Coordination and Orchestration 9

Roberto Bruni @ IMT Lucca 23 March 2005 Flow Composition A 2 A 3 A 1 A 4 A 5 Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 10

Roberto Bruni @ IMT Lucca 23 March 2005 Transactional Flows A 2 A 3 A 1 A 4 A 5 Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 11

Roberto Bruni @ IMT Lucca 23 March 2005 Compensation Activities A 1 A 2 A 3 B 2 B 3 B 1 A 4 B 4 A 5 B 5 Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 12

Roberto Bruni @ IMT Lucca 23 March 2005 Compensation Flow A 1 A 2 A 3 B 2 B 3 B 1 A 4 B 4 A 5 B 5 Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 13

Roberto Bruni @ IMT Lucca 23 March 2005 Nested Flow Diagrams A 1 A 2 A 3 B 2 B 3 B 1 A 4 B 4 P Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 14

Roberto Bruni @ IMT Lucca 23 March 2005 Approaches n Interaction based Composition, Conversational Patterns or Global Model n n Services describe the ways they can be engaged in a larger process Flow Composition or Hierarchical Patterns n Institutions Markets Technologies IMT Similar to workflow systems: a process describes the flow of both control and data among WS Models and Languages for Coordination and Orchestration 15

Roberto Bruni @ IMT Lucca 23 March 2005 n The Road to Compensating CSP First use of process algebras for modeling compensations: n St. AC (Structured Activity Compensation) n n n by Michael Butler, Carla Ferreira et al. poorly disciplined, several variants / improvements along the years Compensating CSP n n n by Michael Butler, Carla Ferreira, Tony Hoare robust formalization of compensable flow compositions closer to the spirit of Process Algebras n n Institutions Markets Technologies IMT few key primitives inspired by BPEL n and to the independently developed saga calculus Models and Languages for Coordination and Orchestration 16

Roberto Bruni @ IMT Lucca 23 March 2005 n Alphabet of observable actions n n ranged by A, B, . . . Set of special events = { , !, ? } n n n Compensating CSP: Ingredients ranged by disjoint from Interactive processes n Standard processes n n Compensable processes n Institutions Markets Technologies IMT ranged by P, Q, . . . ranged by PP, QQ, . . . Models and Languages for Coordination and Orchestration 17

Roberto Bruni @ IMT Lucca 23 March 2005 Compensating CSP: Syntax Standard processes Compensable processes atomic action P, Q : : = A transaction block PP, QQ : : = P % Q compensation pair | [ PP ] choice | P+Q | PP + QQ sequential composition | P; Q | PP ; QQ parallel composition | P || Q | PP || QQ normal termination | SKIPP | THROW throw an interrupt | THROWW | YIELD yield to an interrupt | YIELDD | P Q interrupt handler Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 18

Roberto Bruni @ IMT Lucca 23 March 2005 Compensating CSP: Example Order. Transaction = [ Process. Order ] Process. Order = (Accept. Order % Restock. Order) ; Fulfill. Order = Book. Courier % Cancel. Courier || Pack. Order || Credit. Check ; ( Ok ; SKIPP + Not. Ok ; THROWW ) Pack. Order = ||i Items ( Pack. Item(i) % Unpack. Item(i) ) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 19

Roberto Bruni @ IMT Lucca 23 March 2005 n Compensating CSP: Semantics Denotational Trace Semantics n n defined in a compositional style each standard process is assigned a set of traces n n traces are ranged over by p, q, . . . they are -event-terminated sequences of actions n n each compensable process is assigned a set of tracepairs (p, q) where n n Institutions Markets Technologies IMT ex. A, B, p is the forward trace q is the corresponding compensation trace Models and Languages for Coordination and Orchestration 20

Roberto Bruni @ IMT Lucca 23 March 2005 n Compensating CSP: Ordinary Traces All traces for standard processes have three possible shapes: n A, B, n n A, B, ! n n Institutions Markets Technologies IMT trace leading to interrupt yield pq denotes the trace obtained by juxtaposition n n trace leading to interrupt throw A, B, ? n n trace leading to normal termination ex. A B, = A, B Note: n unlike trace semantics for CSP, prefix traces are not considered Models and Languages for Coordination and Orchestration 21

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Atomic Actions For any A we define A =trace { A, } (The process performs a single atomic event and terminates successfully) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 22

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Skip SKIP =trace { } (SKIP immediately terminates successfully) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 23

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Throw THROW =trace { ! } (THROW immediately raises an interrupt) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 24

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Yield YIELD =trace { , ? } (YIELD can either yield to a raised interrupt or terminate) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 25

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Choice For any standard processes P and Q P + Q =trace P Q (The semantics for choice is the union of the possible traces of P and Q) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 26

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Sequential Composition For any standard processes P and Q P ; Q =trace { p; q | p P q Q } where the sequential operator is defined on traces by p ; q = pq p ; q = p if Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 27

Roberto Bruni @ IMT Lucca 23 March 2005 SECOND HOMEWORK: Prove or Disprove That: For any standard processes P, Q, R, S P; (Q+R) =trace (P; Q)+(P; R) (P+Q); R =trace (P; R)+(Q; R) (P+Q); (R+THROW) =trace P+Q (P; Q); R =trace (P; Q); R P; SKIP =trace P SKIP; P =trace P P; THROW =trace THROW; P =trace THROW YIELD; YIELD =trace YIELD Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 28

Roberto Bruni @ IMT Lucca 23 March 2005 n Proof by obviousness n n "All in favor? . . . " Proof by majority rule n n "Because of the time constraint, I leave the proof to you. " Proof by general agreement n n "The proof is so clear that it need not be mentioned. " Proof by lack of sufficient time n n Playful Digression: 10 Advanced Proof Methods I Only to be used if general agreement is impossible Proof by accident n Institutions Markets Technologies IMT "Hey, what have we here? !" Models and Languages for Coordination and Orchestration 29

Roberto Bruni @ IMT Lucca 23 March 2005 n Proof by authority n n "Don't be stupid; of course it's true. " Proof by terror n n "I just have this gut feeling. . . " Proof by intimidation n n "Well, Don Knuth says it's true, so it must be!" Proof by intuition n n Playful Digression: 10 Advanced Proof Methods II When intimidation fails. . . Proof by deception n Institutions Markets Technologies IMT "Now everyone turn their backs. . . " Models and Languages for Coordination and Orchestration 30

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Interrupt Handler For any standard processes P and Q P Q =trace { p q | p P q Q } where the interrupt handling is defined on traces by p ! q = pq p q = p if ! Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 31

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Useful Laws for Interrupt For any standard processes P, Q, R (P Q) R =trace P (Q R) SKIP P =trace SKIP YIELD P =trace YIELD THROW P =trace P Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 32

Roberto Bruni @ IMT Lucca 23 March 2005 n n n Synchronous execution of observable actions is not supported (yet) Concurrency modeled by interleaving Processes running in parallel can synchronise only n n n Compensating CSP: A Limitation on joint termination or on joint interruption The kind of synchronization is decided by just looking at the terminal events of parallel traces n Institutions Markets Technologies IMT it is convenient to define an operator for deriving the joint terminal event Models and Languages for Coordination and Orchestration 33

Roberto Bruni @ IMT Lucca 23 March 2005 Compensating CSP: Joint Terminal Event Let 1 and 2 be the terminal events of two parallel traces Their joint terminal event 1& 2 is defined by 1 2 1& 2 (& is commutative) ! ! ! Institutions Markets Technologies IMT ! ? ! ! ! ? ? ? Models and Languages for Coordination and Orchestration 34

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Parallel Composition For any standard processes P and Q P || Q =trace { r | r p||q p P q Q } where p 1 || q 2 = { r 1& 2 | r int(p, q) } int(p, ) = int( , p) = {p} int( A p, B q) = { A r | r int(p, B q) } { B r | r int( A p, q) } Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 35

Roberto Bruni @ IMT Lucca 23 March 2005 Trace Semantics: Useful Laws for Parallel For any standard processes P, Q, R P || Q =trace Q || P (P || Q) || R =trace P || (Q || R) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 36

Roberto Bruni @ IMT Lucca 23 March 2005 Under Which Circumstances? P || Q =trace P; Q + Q; P THROW || (YIELD ; P) =trace THROW + P; THROW Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 37

Roberto Bruni @ IMT Lucca 23 March 2005 Trace-Pair Semantics: Compensable Choice For any compensable processes PP and QQ PP + QQ =trace PP QQ (The semantics for choice is the union of the possible traces of PP and QQ) Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 38

Roberto Bruni @ IMT Lucca 23 March 2005 Trace-Pair Semantics: Compensable Parallel For any compensable processes PP and QQ PP || QQ =trace { z | z x||y x PP y QQ } where (p, p') || (q, q') = { (r, r') | r p||q r' p'||q' } Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 39

Roberto Bruni @ IMT Lucca 23 March 2005 Trace-Pair Semantics: Compensable Sequential For any compensable processes PP and QQ PP ; QQ =trace { x; y | x PP y QQ } something wrong? where (p , p') ; (q, q') = (pq, q'; q) (p , p') ; (q, q') = (p , p') if Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 40

Roberto Bruni @ IMT Lucca 23 March 2005 Trace-Pair Semantics: Compensation Pair For any standard processes P and Q P % Q =trace { p%q | p P q Q } { ( ? , ) } where p % q = (p , q) p % q = (p , ) if Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 41

Roberto Bruni @ IMT Lucca 23 March 2005 Trace-Pair Semantics: Compensable Basic Processes SKIPP =trace SKIP % SKIP THROWW =trace THROW % SKIP YIELDD =trace YIELDD % SKIP Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 42

Roberto Bruni @ IMT Lucca 23 March 2005 Trace-Pair Semantics: Transaction Block For any compensable process PP [PP] =trace { pp' | (p ! , p') PP } { p | (p , p') PP } Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 43

Roberto Bruni @ IMT Lucca 23 March 2005 Trace-Pair Semantics: Other Useful Laws For any composable processes PP; QQ and for any standard processes P, Q Institutions Markets Technologies IMT PP || QQ =trace QQ || PP (PP || QQ) || RR =trace PP || (QQ || RR) (PP ; QQ) ; RR =trace PP ; (QQ ; RR) PP ; SKIPP =trace SKIPP ; PP THROWW ; PP =trace THROWW YIELDD ; (P % Q) =trace (P % Q) Models and Languages for Coordination and Orchestration 44

Roberto Bruni @ IMT Lucca 23 March 2005 THIRD HOMEWORK Evaluate the semantics of [ THROWW ] =trace ? ? ? [ YIELDD ] =trace ? ? ? A % A' ; B % B' =trace ? ? ? Prove that: if P, P', Q, Q' terminate successfully, neither raising nor yielding to interrupts, then [ P%P' || Q%Q' || THROWW ] =trace Institutions Markets Technologies IMT SKIP + (P; P') + (Q; Q') + (P||Q); (P'||Q') Models and Languages for Coordination and Orchestration 45

Roberto Bruni @ IMT Lucca 23 March 2005 Is the Semantics Adequate? n Formal definitions can n n The semantics we have seen n lead to conceptual clarifications make emerge submersed aspects give insights for language design is compositional, simple and intuitive models forward and backward flows Is it the appropriate one? n n Institutions Markets Technologies IMT Does it match our intuition? Any other option available? Models and Languages for Coordination and Orchestration 46

Roberto Bruni @ IMT Lucca 23 March 2005 Semantics vs Intuition n Expectations: n n if no interrupt is raised a "maximal" forward flow should be executed if an interrupt is raised, n n n all previously-completed activities are compensated no consequent activities are executed within that transaction if an interrupt is raised during the backward flow? what else? How can we reasonably conclude that the semantics is "correct"? Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 47

Roberto Bruni @ IMT Lucca 23 March 2005 Some Ideas I n n P = { p | p P } P A successful trace is n either a forward trace with possibly many successful transactions n n or a trace with possibly many successfully compensated transactions n Institutions Markets Technologies IMT PP = { p | (p , q) PP } PP = { pq | (p ! , q ) PP } Models and Languages for Coordination and Orchestration 48

Roberto Bruni @ IMT Lucca 23 March 2005 Some Ideas II n Theory of "cancellation" n n each (forward) A has a compensating (backward) A° AA° is "essentially" SKIP n n but A°A is not SKIP Theory of "independence" n independent actions can occur in either order n n independency, written , is symmetric n n Institutions Markets Technologies IMT ex. parallel actions A||B unlike cancelling independency can be exploited to bring A and A° closer Models and Languages for Coordination and Orchestration 49

Roberto Bruni @ IMT Lucca 23 March 2005 Some Ideas III n Abstract effect n n A° annihilates A if we remove all such pairs from a trace, we are left with an abstract residual A(s) n n it gives the meaningful actions performed Take s * and define recursively n A(s) = A(pqr) n n Institutions Markets Technologies IMT if s = p A q A° r and B q. B A° A(s) = s n otherwise Models and Languages for Coordination and Orchestration 50

Roberto Bruni @ IMT Lucca 23 March 2005 Example: Cancellation Assume A° B°, then: A( A, B, C, C°, A°, B°, ) = A( A, B, A°, B°, ) = A( A, A°, ) = A( ) = Institutions Markets Technologies IMT Models and Languages for Coordination and Orchestration 51

Roberto Bruni @ IMT Lucca 23 March 2005 Adequacy Criterion I n When considering [PP] one would like that n for any p [PP] n n n this is equivalent to require that n n n if p is a forward trace, then A(p) = p if p is a compensated trace, then A(p) = p PP . A(p) = p p PP . A(p) = Is this assumption strong enough? n Institutions Markets Technologies IMT If we know that it holds for PP, what can we say about [ PP ; THROWW ]? Models and Languages for Coordination and Orchestration 52

Roberto Bruni @ IMT Lucca 23 March 2005 Adequacy Criterion II n PP is called self-cancelling, written S(PP) if n n In other words, n n (p , p' ' ) PP. A(pp' ' ) = we must guarantee that in any execution of PP, even partial ones, the installed compensation is able to cancel the actions executed so far We can then prove interesting properties n ex. S(PP) implies A([PP; THROWW]) =trace SKIP n Institutions Markets Technologies IMT where A(. ) is extended element-wise to set of traces Models and Languages for Coordination and Orchestration 53

Roberto Bruni @ IMT Lucca 23 March 2005 A Well-Behaving Fragment n We would like to characterize syntactically a fragment of compensable processes guaranteeing the self-cancelling property n it is easy to check that: n n n Institutions Markets Technologies IMT S(SKIPP), S(THROWW) and S(YIELDD) if S(PP) and S(QQ), then S(PP+QQ) if S(PP) and S(QQ), then S(PP; QQ) But what can we say about PP||QQ ? And what about P%Q? Models and Languages for Coordination and Orchestration 54

Roberto Bruni @ IMT Lucca 23 March 2005 Restriction on Parallel n When trying to prove that n n a difficulty emerges due to interleaving n n n if S(PP) and S(QQ), then S(PP||QQ) in the resulting traces A and A° from PP are interleaved with actions from QQ and vice versa To prove thesis we must assume that: n Institutions Markets Technologies IMT A° B° for all A° in PP and B° in QQ Models and Languages for Coordination and Orchestration 55

Roberto Bruni @ IMT Lucca 23 March 2005 n P%Q provides n nesting of compensations programmable compensations These features are indeed complex ones n n Restriction on Compensation Pairs their foundations are under investigation A safe assumption is n allowing only A%A° in place of P%Q n Institutions Markets Technologies IMT Then S(A%A°) trivially holds Models and Languages for Coordination and Orchestration 56