Oblivious AQM and Nash Equilibria D Dutta A

Oblivious AQM and Nash Equilibria D. Dutta, A. Goel and J. Heidemann USC/ISI IEEE INFOCOM 2003 - The 22 nd Annual Joint Conference of the IEEE Computer and Communications Societies Presented By Sharon Mendel Game Theory in Networks Seminar 25/01/2006 Oblivious AQM & Nash Equilibria

Before we Begin… Today’s Internet - Motivation Oblivious AQM & Nash Equilibria

Before we Begin… Active Queue Management ? ? A congestion control protocol (e. g. TCP) operates at the end-points and uses the drops or marks received from the Active Queue Management policies (e. g. Droptail, RED) at routers as feedback signals to adaptively modify the sending rate in order to maximize its own goodput. Oblivious AQM & Nash Equilibria 3

Before we Begin… Transport Control Protocol TCP is the dominating transport layer protocol in the Internet and accounts for over 90% of the total traffic. The TCP Protocol is well defined, robust and congestion-reactive (thus stable). The end-to-end congestion control mechanisms of TCP have been a critical factor in the robustness of the Internet. It is widely believed that if all users deployed TCP, networks will rarely see congestion collapses and the overall utilization of the network will be high. Oblivious AQM & Nash Equilibria 4

Before we Begin… Today’s Internet There are indications that the amount of noncongestion-reactive traffic is on the rise. n n Most of this misbehaving traffic does not use TCP. e. g. Real media, network games, other real time multimedia applications. The unresponsive behavior can result in both unfairness and congestion collapse for the Internet. The network itself must now participate in controlling its own resource utilization. * Some of the previous slides: “Promoting the Use of End-to-End Congestion Control in the Internet”, S. Floyd and K. Fall, IEEE/ACM Transactions on Networking Vol. 7 1999. Oblivious AQM & Nash Equilibria 5

Before we Begin… The Paper’s Motivation TCP (and in fact, any transport protocol) does not guarantee good performance in the face of aggressive, greedy users (who are willing to violate the protocol to obtain better performance). Protocol Equilibrium – A protocol which leads to an efficient utilization and a somewhat fair distribution of network resources (like TCP does), and also ensure that no user can obtain better performance by deviating from the protocol. If protocol equilibrium is achievable, then it would be a useful tool in designing robust networks. Oblivious AQM & Nash Equilibria 6

More Introduction Oblivious AQM and Nash Equilibria Oblivious AQM & Nash Equilibria

More Introduction Oblivious AQM Oblivious (stateless) AQM scheme – a router strategy that does not differentiate between packets belonging to different flows. Stateful schemes – e. g. Fair-Queuing. Stateful Schemes offer good performance, but oblivious schemes are easier to implement. Oblivious AQM & Nash Equilibria 8

More Introduction Popular AQM Schemes (1) Congestion avoidance is achieved through packet dropping. 1. Drop-Tail – Buffers as many packets as it can and drops the ones it can't buffer: n n Distributes buffer space unfairly among traffic flows. Can lead to global synchronization as all TCP connections "hold back" simultaneously, DT hence networks become under-utilized. Oblivious AQM & Nash Equilibria 9

More Introduction Popular AQM Schemes (2) 2. RED – Random Early Dedication - Monitors the average queue size and drops packets based on statistical probabilities: n n n If the buffer is almost empty, all incoming packets are accepted; As the queue grows, the probability for dropping an incoming packet grows; When the buffer is full, the probability has reached 1 and all incoming packets are dropped. Considered more fair than tail drop - The more a host transmits, the more likely it is that its packets are dropped. Prevents global synchronization and achieves lower average buffer occupancies. R ED Oblivious AQM & Nash Equilibria 10

More Introduction Oblivious AQM and Nash Equilibria The paper studies the existence and quality of Nash equilibria imposed by oblivious AQM schemes on selfish agents: 1. Existence 2. Efficiency 3. Achievability Oblivious AQM & Nash Equilibria 11

Content üIntroduction The Model Existence Efficiency Achievability Summary and Future Work Oblivious AQM & Nash Equilibria 12

The Model The Markovian Internet Game Oblivious AQM & Nash Equilibria

The Model The Internet Game Players – The end-to-end selfish traffic agents. n Each player has a strategy which is to control the average rate he tries to push through the network. n Users’ Performance Metric – goodput. Rules – set by the AQM policies (AQM schemes in routers). Nash Equilibrium – No selfish agent has any incentive to unilaterally deviate from it’s current state. Oblivious AQM scheme leads to a protocol equilibrium only if it imposes a Nash equilibrium on the selfish users. Paper’s focus – AQM schemes that guarantee bounded average buffer occupancy regardless of the total arrival rate. Oblivious AQM & Nash Equilibria 14

The Model The Markovian Internet Game The agents generate Poisson traffic. n n Does not accurately model Internet traffic, yet a reasonable first step. Each user controls its own offered load = the average Poisson traffic rate. The system is modeled as a M/M/1/K queue: n n Average service time - Without loss of generality assumed to be unity. Buffers capacity - K=B. No assumptions are made on the selfish protocol (i. e. TCP, AIMD etc’). Oblivious AQM & Nash Equilibria 15

The Model The M/M/1/K Internet Game n – Number of users – players. λi – The Poisson average arrival rate of player i. Ui – Utility function of player i. μi – Goodput , Ui = μi. p – AQM router drop probability due an average aggregated load (offered load) of λ and an average service time of unity. A symmetric Nash equilibrium - Ensures that every agent has the same goodput at equilibrium. Unless mentioned otherwise, quantities such as the rates, goodput and throughput are averages (Poisson traffic sources). Oblivious AQM & Nash Equilibria 16

DT The Model R ED Nash Equilibrium Conditions (1) 1. No agent can increase their goodput, at Nash equilibrium, by either increasing or decreasing their throughput: 2. A symmetric Nash equilibrium: 3. Oblivious AQM scheme, hence functions of router state (drop probability, queue length) are independent in i: Utility Function in Nash Equilibrium : Nash Equilibrium Condition : Nash Equilibrium Satisfying Condition – Necessary and sufficient Oblivious AQM & Nash Equilibria 17

The Model Nash Equilibrium Conditions (2) λNash , μNash , p. Nash - The aggregate throughput (offered load), goodput, drop probability respectively. The Nash equilibrium imposed by an AQM scheme is efficient if the goodput of any selfish agent is bounded below when the throughput (offered load) of the same agent is bounded above: Nash Equilibrium Efficiency Condition : c 1 c 2 some constants p. Nash is bounded Oblivious AQM & Nash Equilibria 18

Existence Are there oblivious AQM schemes that impose Nash equilibria on selfish users? Oblivious AQM & Nash Equilibria

Existence Drop-Tail Queuing p - Drop Probability = Probability to find a full system. From Queuing Theory: Theorem 1: There is NO Nash Equilibrium for selfish agents and routes implementing Drop-Tail queuing. QED. Proof: applying the condition for Nash equilibrium: Oblivious AQM & Nash Equilibria 20

Existence Random Early Detection Approximated steady state RED model: “Faster network simulation with scenario pre-filtering” Tech. Rep. , USC/ISI Tech Report 550, November 2001. lq – Queue length p – Drop Probability λ – Aggregated Offered Load 0 p= If lq <minth If 1 minth lq maxth Otherwisw From Queuing Theory, at steady state: RED Router, at steady state: lq maxth Steady State: Oblivious AQM & Nash Equilibria 21

Existence RED and Nash Equilibria Theorem 2: RED Does NOT impose a Nash equilibrium on uncontrolled selfish agents. Proof: applying the condition for Nash equilibrium: QED. RED is oblivious: Summary: n n RED punishes all flows with the same drop probability. The nature of the drop function is considerably gentle. Misbehaving flows can push more traffic and get less hurt (marginally). There is no incentive for any source to stop pushing packets. Oblivious AQM & Nash Equilibria Nash Equlibria does not exist. 22

Existence Virtual Load RED model: lvq – M/M/1 Queue length when facing the same load : 0 p= If lvq <minth If 1 minth lvq maxth Otherwise Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if Proof: Throughput at Nash equilibrium is independent of minth: Oblivious AQM & Nash Equilibria 23

Efficiency If an Oblivious AQM scheme can impose a Nash equilibria, is that equilibria efficient, in terms of achieving high goodput and low drop probability? Oblivious AQM & Nash Equilibria

Efficiency Theorem 4: VLRED is not efficient imposing a Nash Equilibrium on selfish agents Normalized Data Example - Efficiency of VLRED Proof: Applying Nash equilibrium satisfying and efficiency conditions: α, β - some constants Total Offered Load Goodput minth = 0 maxth =50 n – 1, . . , 50 Number Of Flows nμ = θ(lvq 2) lvq 2 nμ/α The goodput falls to zero asymptotically. QED. Oblivious AQM & Nash Equilibria 25

Efficiency Efficient Nash AQM scheme Total Offered Load Normalized Data Assume – Total desirable offered load at Nash equilibrium: ENAQM drop probability is bounded Goodput Drop Probability Number Of Flows Oblivious mechanism can ensure an efficient Nash equilibria under selfish behavior of users. Oblivious AQM & Nash Equilibria 26

Achievability How easy is it for players (users) to reach the equilibrium point? or How can we ensure that agents actually reach the Nash equilibrium state? Oblivious AQM & Nash Equilibria

Ensuring a Nash equilibrium by an Oblivious AQM Achievability λ*i – Offered load at equilibria when the number of agents is i. Δi = λ*i - λ*i-1 = iα α some constant p = ƒ(λ*i) non decreasing and convex. Applying Nash equilibrium satisfying and efficiency conditions: c 1, c 2 - some constants From convexity of p*i : From efficiency: c - constant The sensitivity coefficient falls faster than the inverse quadric. The equilibrium imposed by any oblivious AQM strategy is (very) sensitive to the number of agents, thus making it impractical to deploy in the Internet. Agents need the help of the router to reach the equliibria. Oblivious AQM & Nash Equilibria 28

Summary and Future Work w. Overview w. Now what? – Future Work w. Some further work Oblivious AQM & Nash Equilibria

Summary and Future Work Overview Introduction – Today’s Internet. The proposed model – Markovian (M/M/1/K) Game Existence – n n Drop tail and RED cannot impose a Nash equilibra. VLRED imposes a Nash equilibra. But the equilibrium points do not have a very high utilization. Efficiency - ENAQM imposes an efficient Nash equilibra. Achievability - Equilibrium points in oblivious AQM strategies are very sensitive to the change in the number of users. n It may be hard to deploy oblivious schemes that do have Nash equilibria without the explicit help of a protocol. Oblivious AQM & Nash Equilibria 30

Summary and Future Work Now What ? - Future Work VLRED – Explore why the Nash equilibria do not result in good network utilization. n Conjecture – VLRED’s drop function becomes very harsh as we reach equilibria. Study gentler versions of VLRED and determine whether such modification can still impose Nash equilibria. Design protocols which lead to efficient network operation, such that no user has any incentive to unilaterally deviate from the protocol – can it be done ? – The Protocol Equilibrium Question. Oblivious AQM & Nash Equilibria 31

Summary and Future Work Some Further work (1) “Towards Protocol Equilibrium with Oblivious Routers” D. Dutta, A. Goel and J. Heidemann, IEEE INFOCOM 2004. n “…In this paper, we show that if routers used EWMA to measure the aggregate rate, then the best strategy for a selfish agent to minimize its losses is to arrive at a constant rate. Even though the protocol space is arbitrary, our scheme ensures that the best greedy strategy is simple, i. e. send with CBR. Then, we show we can use the results of an earlier paper to enforce simple and efficient protocol equilibria on selfish traffic agents…” Oblivious AQM & Nash Equilibria 32

Summary and Future Work Some Further work (2) “Pricing Differentiated Services A Game -Theoretic Approach”, E. Altman, D. Barman, R. El Azouzi, D. Ros, B. Tuffin, (accepted for publication in Computer Networks, 2005). n n “…The goal of this paper is to study pricing of differentiated services and its impact on the choice of service priority at equilibrium. We consider both TCP connections as well as non controlled (real time) connections…. We first study the performance of the system as a function of the connections’ parameters and their choice of service classes. We then study the decision problem of how to choose the service classes. We model the problem as a noncooperative game. We establish conditions for an equilibrium to exist and to be uniquely defined. We further provide conditions for convergence to equilibrium from non equilibria initial states. We finally study the pricing problem of how to choose prices so that the resulting equilibrium would maximize the network benefit…” “…The paper (“Oblivious AQM and Nash Equilibria”) restricted itself to symmetric users and symmetric equilibria and the pricing issue was not considered. In this framework, with a common RED buffer, it was shown that an equilibrium does not exist. An equilibrium was obtained and characterized for an alternative buffer management that was proposed, called VLRED. We note that in contrast to (Oblivious AQM and Nash Equilibria”), since we also include in the utility of CBR traffic a penalty for losses we do obtain an equilibrium when using RED…” Oblivious AQM & Nash Equilibria 33

Thank You ! Oblivious AQM & Nash Equilibria

Appendixes w. Stateful Schemes – Fair Queuing w. From Queuing Theory w. Nash Equilibrium Conditions w. Approximated Steady State RED Model w. VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof w. Efficient Nash AQM Drop Probability Oblivious AQM & Nash Equilibria

Introduction - Appendix Stateful Schemes – Fair Queuing - Nagle’s Algorithm Gateways maintain separate queues for packets from each individual source. The queues are serviced in a round-robin manner. Nagle’s algorithm, by changing the way packets from different sources interact, does not reward, nor leave others vulnerable to, anti-social behavior. “Analysis and Simulation of a Fair Queuing Algorithm”, A. Demers, S. Keshav and S. J. Shenker, ACM SIGCOMM, 1989. Oblivious AQM & Nash Equilibria 36

From Queuing Theory M/M/1/K Queuing system - Poisson arrivals, Exponentially distributed λ λ buffer: λ serviceλ times, oneλ server and finite capacity 0 μ K 1 2 1 μ μ μ K μ PASTA - Poisson Arrivals See Time Averages – For a queuing system, when the arrival process is Poisson and independent of the service process: The probability that an arriving customer finds i customers in the system Equals The probability that the system is at state i. πi – Probability that the system is in state i, Using birth-death model: Block Probability Oblivious AQM & Nash Equilibria 37

The Model – Appendix Nash Equilibrium Condition 1. 2. 3. 4. 5. Substituting 2, 3 in. Satisfying 5 we obtain: Nash Equilibrium Condition : Oblivious AQM & Nash Equilibria 38

Existence - Appendix Approximated Steady State RED Model Approximated RED Transfer Function Drop prob. “Faster network simulation with scenario pre-filtering” Tech. Rep. , USC/ISI Tech Report 550, November 2001. Example: minth = 10 maxth =20 n – 1, . . , 50 Pmax = 0. 3 Normalized Offered Load Avg. Buffer Occupancy Drop prob. Standard RED Transfer Function Approximated RED Buffer Occupancy Avg. Buffer Occupancy Oblivious AQM & Nash Equilibria Normalized Offered Load 39

Existence - Appendix VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (1) Virtual Load RED model: lvq – M/M/1 Queue length when facing the same load : 0 p= If lvq <minth If 1 minth lvq maxth Otherwise Proof of Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if n n Assume the drop probability can be written as a continues function for all lvq<maxth: Nash equilibrium condition: Oblivious AQM & Nash Equilibria 40

Existence - Appendix VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (2) Substituting This is true if Oblivious AQM & Nash Equilibria and simplifying we obtain: (by Substituting n=1) 41

Efficient Nash AQM Drop Probability Efficiency - Appendix Total desirable offered load at Nash equilibrium: Substituting y 2=1 -λ and then integrating: k - arbitrary constant to be determined k is determined so when there is one user, the drop probability is zero as long as his offered load is less than unity The offered load at Nash equilibria is bounded drop probability at equilibria is bounded (proved by substitution). Oblivious AQM & Nash Equilibria 42