IEEE International Conference on Service Computing 2007 RegretBased

IEEE International Conference on Service Computing 2007 Regret-Based Decentralized Adaptation of Web Processes with Coordination Constraints Authors: Yunzhou Wu, Prashant Doshi Speaker: Haibo Zhao LSDIS Lab, Dept. of Computer Science, University of Georgia

OUTLINE Ø Introduction Ø Related work Ø Background v. Framework v. Markov decision process v. Multi-Agent MDP model (M-MDP) v. Simple decentralized MDP model (MDP-Co. M) Ø Regret-based decentralized approach Ø Empirical results Ø Discussion

Introduction Ø Web process adaptation problem v. Adaptation to atypical input data (exogenous events) v. Constraints exist between disparate components of the process Ø Challenge v. Optimally adapt to exogenous events while respecting the inter-service constraints Ø An Example: Trip Planner process

Motivating Scenario coordination constraint Trip Planner coordination constraint v. A change in the date of departure requires rebooking for all three activities v. A change in the destination airport requires modifications to two activities

Related Work Ø Verification of correctness of manual changes to control flow v. Adept (Reichert&Dadam 98) v. Workflow inheritance (Aalst&Basten 02) v. Inter-task dependencies (Attie et al. 93) Ø Event Condition Action (ECA) rules for adaptation v. Agentwork (Muller et al. 04) Ø Change of service providers based on migration rules in E-Flow v(Casati et al. 00)

Background Ø Framework Ø Markov decision process (MDP) Ø Simple approaches to adaptation (Verma et al ICWS 2006 v. Multi-agent MDP model (M-MDP) v. Decentralized MDP model with simple coordination mechanism (MDP-Co. M) )

Framework formulized as MDP

MDP Model Ø Models sequential decision making under uncertainty Ø MDP is a tuple: <S , PA, T, C, OC> v S: state space The set of states of the process v PA: permissible actions for any given states Invoking a WS or other actions a SM could take v T: transition function, Pr(s’|s, a) The probability of the resulting state on performing a permissible action from some local state v C: the cost of action Cost of invoking a WS could be elicited from WS descriptions v OC: optimality criterion Finite horizon

MDP Model Ø MDP gives a policy which designates an optimal action for a given state Ø The policy guarantees a minimal long–term expected cost Ø In our approach, we use policies to guide the behavior of service manages

Simple Approaches for Adaptation ØM-MDP Ø Globally optimal Ø Does not scale efficiently for large processes v Complexity exponential in the number of SMs

Simple Approaches for Adaptation Ø MDP-Co. M Model v. Every SM makes its own decision v. Adopts a simple coordination mechanism (Co. M) v. Co. M is a Finite State Machine (FSM) v. Each SM can observe its own state and Co. M state perfectly, but no states of other SMs Intent U * C

Simple Approaches for Adaptation Ø MDP-Co. M (Cont’d) Ø Expanded Model v S’=S x Y, Y={U, C}, state space of Co. M v A’=A U {intent} Ø If SMi intends to take a coordinating action C, other SMs have to follow Ø Action C may be the best for SMi, but may not be optimal for all other SMs Ø Compromised global optimality v Some SMs are forced to take sub-optimal coordinating action Ø Scales well to large number of SMs

Regret-based Decentralized Adaptation Ø Problems of previous approaches v. M-MDP Does not scale well to large number of SMs Needs a process manager to oversee all SMs v. MDP-Co. M Compromised global optimality Co. M should be observable by all SMs

Regret-based Decentralized Adaptation Ø Regret-based approach (MDP-R-Co. M) ü Scales well to large number of SMs as MDP-Co. M ü Better optimality than MDP-Co. M ü No central component required

MDP-R-Co. M communication channel

MDP-R-Co. M Ø V (s) represents the long term expected cost of performing the optimal policy from a given state s Ø Q(s, a) represents the long term expected cost from state s taking action a

MDP-R-Co. M Ø Coordination decision v. MDP-Co. M makes coordination decision based on one SM’s local interest v. MDP-R-Co. M makes coordination decision based on the sum of Q values of SMs

Regret-based Coordination Mechanism SMi ac Q-Req Qj(C) Qj(U) SMj Respond to Q-Req SMk Does not respond to Q-Req

Regret-based Coordination Mechanism

Co. M for SMj that received Q_Req Problem? : Why should SM send Q-value on receiving the Q-value request? v Sending Q-value involves communication cost, its not free!

Regret-based Mechanism Ø SMj will regret not sending Q-value in the case below : v For SMj not sending Q-value, coordination decision is against its local interest v Sending out Q-value would change coordination decision to be in favor of its local interest

Regret-based Mechanism Ø Expected Regret-cost could be calculated given model parameters Ø By not communicating the Q-value, SMj could incur the additional Regret-cost Ø SM will send Q value if v Expected Regret-cost > communication cost of sending Q

Empirical Results v. The plot below evaluate the performance of M-MDP, MDP-R-Co. M, MDP-Co. M, and a random selection approach v. Each data point in the plot is the average of 3, 000 runs in a simulated environment, each run consisted of the SMs performing 20 steps v. The experiment measured the performance of different approaches as the probability of exogenous event (unavailability) varied

Empirical Results Ø M-MDP’s reward performance is best because of its global optimality Ø MDP-R-Co. M performs better than MDP-Co. M because v MDP-R-Co. M make coordination decision based on the sum of Q-Values of all SMs v MDP-Co. M makes coordination decision based on one SM

Empirical Results

Empirical Results The run time of M-MDP increases exponentially to the number of SMs Ø The run time of MDP-Co. M and MDP-R-Co. M increases linearly to the number of SMs Ø The run time of MDP-R-Co. M is a little bit higher than that of MDP-Co. M because of its more sophisticated Co. M

Discussion & Future Work Ø MDP-R-Co. M ü Adopts a regret-based adaptation mechanism ü Does not need any central component ü Performs significantly better than MDP-Co. M while scaling well to large process Ø Future work v MDP-R-Co. M relies on a communication channel between SMs v Other approaches are needed when it is hard to implement these communication channels

Thanks Questions?