Introduction to Game Theory Seminar Lecture 1 Daniel

Introduction to Game Theory Seminar Lecture 1 Daniel R. Figueiredo (EPFL) March 2006

What is Game Theory About? r Analysis of situations where conflict of interests are present 2 2 r Game of Chicken m driver who steers away looses r What should drivers do? r Goal is to prescribe how conflicts can be resolved

Applications of Game Theory r Theory developed mainly by mathematicians and economists m contributions from biologists r Widely applied in many disciplines m from economics to philosophy, including computer science (AI) m goal is often to understand some phenomena r “Recently” applied to computer networks m Nagle, RFC 970, 1985 • “datagram networks as a multi-player game” m wider interest starting around 2000

Limitations of Game Theory r No unified solution to general conflict resolution r Real-world conflicts are complex m models can at best capture important aspects r Players are considered rational m determine what is best for them given that others are doing the same r No unique prescription m not clear what players should do r But it can provide intuitions, suggestions and partial prescriptions m the best mathematical tool we have

What is a Game? r A Game consists of m at least two players m a set of strategies for each player m a preference relation over possible outcomes r Player is general entity m individual, company, nation, protocol, animal, etc r Strategies m actions which a player chooses to follow r Outcome m determined by mutual choice of strategies r Preference relation m modeled as utility (payoff) over set of outcomes

Classification of Games r Many types of games m three major categories r Non-Cooperative (Competitive) Games m individualized play, no bindings among players r Cooperative Games m play as a group, possible bindings r Repeated and Evolutionary Games m dynamic scenario

Matrix Game (Normal form) Strategy set for Player 1 Strategy set for Player 2 A B C A (2, 2) (0, 0) (-2, -1) B (-5, 1) (3, 4) (3, -1) Payoff to Player 1 Payoff to Player 2 r Simultaneous play m players analyze the game and then write their strategy on a piece of paper

More Formal Game Definition r Normal form (strategic) game ma finite set N of players m a set strategies for each player m payoff function for each player • where chosen by all players is the set of strategies r A is the set of all possible outcomes r is a set of strategies chosen by players m defines r an outcome

Two-person Zero-sum Games r One of the first games studied m most well understood type of game r Players interest are strictly opposed m what one player gains the other loses m game matrix has single entry (gain to player 1) r Intuitive solution concept m players maximize gains m unique solution

Analyzing the Game r Player 1 maximizes matrix entry, while player 2 minimizes Player 1 Strictly dominated strategy (dominated by C) Player 2 A B C D A 12 -1 1 0 B 3 1 4 -18 C 5 2 4 3 D -16 0 1 -1 Weakly dominated strategy (dominated by B)

Dominance r Strategy S strictly dominates a strategy T if every possible outcome when S is chosen is better than the corresponding outcome when T is chosen. r Dominance Principle m rational players never choose dominated strategies r Removal of strictly dominated strategies m iterated removal

Analyzing the Reduced Game Player 2 Player 1 A B D A 12 -1 0 C 5 2 3 D -16 0 -1 Outcome (C, B) seems “stable” m saddle point of game

Saddle Points r An outcome is a saddle point if the outcome is both less than or equal to any value in its row and greater than or equal to any value in its column r Saddle Point Principle m Players should choose outcomes that are saddle points of the game r Value of the game m value of saddle point entry if it exists

Why Play Saddle Points? r If player 1 believes player 2 will play X m player 1 should play best response to X r If player 2 believes player 1 will play Y m player 2 should play best response to Y r Why should player 1 believe player 2 will play X? m playing X guarantees player 2 loses at most v r Why should player 2 believe player 1 will play Y? m playing Y guarantees player 1 wins at least v Powerful arguments to play saddle point

Solving the Game (min-max algorithm) Player 2 Player 1 A A 4 B 3 C 2 D 5 2 B -10 2 0 -1 -10 C 7 5 1 3 1 D 0 8 -4 -5 -5 7 8 2 5 r choose maximum entry in each column r choose the minimum among these r this is the minimax value r choose minimum entry in each row r choose the maximum among these r this is maximin value r if minimax == maximin, then this is the saddle point of game

Multiple Saddle Points r In general, game can have multiple saddle points Player 1 A Player 2 B C D A 3 2 2 5 2 B 2 -10 0 -1 -10 C 5 2 2 3 2 D 8 0 -4 -5 -5 8 2 2 5 r Same payoff in every saddle point m unique value of the game r Strategies are interchangeable m Example: strategies (A, B) and (C, C) are saddle points then (A, C) and (C, B) are also saddle points

Games With no Saddle Points Player 2 Player 1 A B C A 2 0 -1 B -5 3 1 r What should players do? m resort to randomness to select strategies

Mixed Strategies r Each player associates a probability distribution over its set of strategies m players decide on which prob. distribution to use r Payoffs are computed as expectations Player 1 A B 1/3 C 2 -5 2/3 D 0 3 Payoff to P 1 when playing A = 1/3(2) + 2/3(0) = 2/3 Payoff to P 1 when playing B = 1/3(-5) + 2/3(3) = 1/3 r How should players choose prob. distribution?

Mixed Strategies r Idea: use a prob. distribution that cannot be exploited by other player m payoff should be equal independent of the choice of strategy of other player m guarantees minimum gain (maximum loss) r How should Player 2 play? x (1 -x) C D A 2 0 Player 1 B -5 3 Payoff to P 1 when playing A = x(2) + (1 -x)(0) = 2 x Payoff to P 1 when playing B = x(-5) + (1 -x)(3) = 3 – 8 x 2 x = 3 – 8 x, thus x = 3/10

Mixed Strategies r Player 2 mixed strategy m 3/10 C , 7/10 D m maximizes its loss independent of P 1 choices r Player 1 has same reasoning Player 1 x A (1 -x) B Player 2 C D 2 0 -5 3 Payoff to P 2 when playing C = x(-2) + (1 -x)(5) = 5 - 7 x Payoff to P 2 when playing D = x(0) + (1 -x)(-3) = -3 + 3 x 5 – 7 x = -3 + 3 x, thus x = 8/10 Payoff to P 1 = 6/10

Minimax Theorem r Every two-person zero-sum game has a solution in mixed (and sometimes pure) strategies m solution payoff is the value of the game m maximin = v = minimax m v is unique m multiple equilibrium in pure strategies possible • but fully interchangeable r Proved by John von Neumann in 1928! m birth of game theory…

Game Trees (Extensive form) r Sequential play m players take turns in making choices m previous choices are available to players r Game represented as a tree m each non-leaf node represents a decision point for some player m edges represent available choices r Can be converted to matrix game (Normal form) m “plan of action” must be chosen before hand

Game Trees Example Player 1 R L Player 2 Payoff to Player 1 L R 3, -3 0, 0 -2, 2 1, -1 r Strategy set for Player 1: {L, R} r Strategy for Player 2: __, __ what to do when P 1 plays L what to do when P 1 plays R r Strategy set for Player 2: {LL, LR, RL, RR} Payoff to Player 2

More Formal Extensive Game Definition r An extensive form game ma finite set N of players m a finite height game tree m payoff function for each player • where s is a leaf node of game tree r Game tree: set of nodes and edges m each non-leaf node represents a decision point for some player m edges represent available choices (possibly infinite) r Perfect information m all players have full knowledge of game history

Converting to Matrix Game L Player 2 Player 1 R L R 3, -3 0, 0 -2, 2 1, -1 LL LR RL RR L 3 3 0 0 R -2 1 r Every game in extensive form can be converted into normal form m exponential growth in number of strategies

Solving the Game (backward induction) r Starting from terminal nodes m move up game tree making best choice L R L R 3, -3 0, 0 -2, 2 1, -1 Equilibrium outcome L 0, 0 r Saddle point: R -2, 2 P 1 chooses L, P 2 chooses RL Best strategy for P 2: RL Best strategy for P 1: L

Kuhn’s Theorem r Backward induction always leads to saddle point (on games with perfect information) m game value at equilibrium is unique (for zero-sum games) r Consider a modified game of chess m either white wins (+1, -1) m either black wins (-1, +1) r Backward induction on game tree m white has winning strategy no matter what black does m black has winning strategy no matter what white does Chess is a simple game!

Two-person Non-zero Sum Games r Players are not strictly opposed m payoff sum is non-zero Player 2 Player 1 A B A 3, 4 2, 0 B 5, 1 -1, 2 r Situations where interest is not directly opposed m players could cooperate

What is the Solution? r Ideas of zero-sum game: saddle points r mixed strategies r pure strategy equilibrium m no pure strat. eq. Player 2 A B Player 1 A 5, 4 2, 0 B 3, 1 -1, 2 Player A 1 B 5, 0 -1, 4 3, 2 2, 1

Multiple Solution Problem r Games can have multiple equilibria m not equivalent: payoff is different m not interchangeable: playing an equilibrium strategy does not lead to equilibrium Player 2 A B Player 1 A 1, 4 1, 1 B 0, 1 2, 2 equilibria

The Good News: Nash’s Theorem r Every two person game has at least one equilibrium in either pure or mixed strategies r Proved by Nash in 1950 using fixed point theorem m generalized to N person game m did not “invent” this equilibrium concept r Def: An outcome o* of a game is a NEP (Nash equilibrium point) if no player can unilaterally change its strategy and increase its payoff r Cor: any saddle point is also a NEP

The Prisoner’s Dilemma r One of the most studied and used games m proposed in 1950 r Two suspects arrested for joint crime m each suspect when interrogated separately, has option to confess Suspect 2 Q C Suspect 1 Q 2, 2 10, 1 C 1, 10 5, 5 better outcome payoff is years in jail (smaller is better) single NEP

Pareto Optimal r Prisoner’s dilemma: individual rationality Suspect 1 Q C Suspect 2 Q C 2, 2 10, 1 1, 10 5, 5 Pareto Optimal r Another type of solution: group rationality m Pareto optimal r Def: outcome o* is Pareto Optimal if no other outcome is better for all players

Game of Chicken Revisited 2 2 r Game of Chicken (aka. Hawk-Dove Game) m driver who swerves looses Driver 2 swerve stay 0, 0 -1, 5 Driver swerve 1 stay 5, -1 -10, -10 Drivers want to do opposite of one another Will prior communication help?

Game Trees Revisited r Microsoft and Mozilla are deciding on adopting new browser technology (. net or java) m Microsoft moves first, then Mozilla makes its move Microsoft. net java Mozilla. net java 3, 1 1, 0 0, 0 2, 2 r Non-zero sum game m what are the NEP?

NEP and Incredible Threats r Convert the game to normal form Mozilla Microsoft . net java 3, 1 1, 0 0, 0 2, 2 NN NJ JN JJ . net 3, 1 1, 0 java 0, 0 2, 2 r Play “java no matter what” is not credible for Mozilla m if java . net Microsoft plays. net then. net is better for Mozilla than java NEP incredible threat

Removing Incredible Threats and other poor NEP. net r Apply backward induction to game tree r Single NEP remains. net for Microsoft, . net, java for Mozilla . net java 3, 1 1, 0 0, 0 2, 2 . net 3, 1 r In general, multiple NEPs are possible after backward induction m java cases with no strict preference over payoffs java 2, 2

Leaders and Followers r What happens if Mozilla is moves first? Mozilla Microsoft . net java 1, 3 0, 1 0, 0 2, 2 Mozilla: java r NEP after backward induction: Microsoft: . net, java r Outcome is better for Mozilla, worst for Microsoft m incredible threat becomes credible! r 1 st mover advantage m but can also be a disadvantage…

Subgame Perfect Nash Equilibrium r Set of NEP that survive backward induction m in games with perfect information r Def: a subgame is any subtree of the original game that also defines a proper game r Def: a NEP is subgame perfect if its restriction to every subgame is also a NEP of subgame r Thr: every extensive form game with complete information has at least one subgame perferct Nash equilibrium m Kuhn’s theorem, based on backward induction

Weakness of SPNE r Centipede Game two players alternate decision to continue or stop for k rounds m stopping gives better payoff than next player stopping in next round (but not if next player continues) m C S 1, 0 0, 2 C S 3, 1 C S 2, 4 C S 5, 3 C S 4, 6 C C S S 7, 5 6, 8 8, 7 r Backward induction leads to unique SPNE m both players choose S in every turn r Each player believes that the other play will stop the game in next opportunity r How would you play this game with a stranger? m empirical evidence suggests people continue for many rounds

Stackelberg Game r Two players, leader and follower r Two moves, leader then follower m can be modeled by a game tree r Stackelberg equilibrium m Leader chooses strategy knowing that follower will apply best response m this precludes incredible threats r Similar to subgame perfect Nash equilibirum m every Stackelberg equilibrium is also SPNE

Repeated Games r Game played an indefinite number of times m same game, same set of players r Important model in practice m many scenarios repeat themselves r Anomalies of finitely repeated games disappear m cooperation can sometimes emerge! Details to follow in next lecture. . .

Title

Title

Title

Title

Title