Infinitely Repeated Games Finitely Repeated Game Take any

  • Slides: 27
Download presentation
Infinitely Repeated Games

Infinitely Repeated Games

Finitely Repeated Game • Take any game play it, then play it again, for

Finitely Repeated Game • Take any game play it, then play it again, for a specified number of times. • A single play of the game that is repeated is known as the stage game. • Let players observe all previous play. • For every history that you have observed, you could have a different response.

Prisoners’ dilemma 3 times • How many strategies that vary with other players’s actions

Prisoners’ dilemma 3 times • How many strategies that vary with other players’s actions on previous move. First move: C or D Second move: – – • • • C always C if C, D if D D if C, C if D D always Third move: There are 4 possible histories of other guy’s moves. For each move by other guy there are two things you can do on your next move. That gives you 2 x 2 x 2 x 2=16 possible 3 d move strategies. Then you have 2 possible first move strategies, 4 possible 2 d move strategies and 16 possible third move strategies That is 2 x 4 x 16=128 strategies that depend on observed behavior of other guy

Prisoners’ Dilemma Player 2 Cooperate P L A Cooperate y E R 1 Defect

Prisoners’ Dilemma Player 2 Cooperate P L A Cooperate y E R 1 Defect R, R S, T T, S P, P T > R > P > S “Temptation” “Reward” “Punishment” “Sucker”

Twice Repeated Prisoners’ Dilemma Two players play two rounds of Prisoners’ dilemma. Before second

Twice Repeated Prisoners’ Dilemma Two players play two rounds of Prisoners’ dilemma. Before second round, each knows what other did on the first round. Payoff is the sum of earnings on the two rounds.

Two-Stage Prisoners’ Dilemma Working back Player 1 Cooperate Defect Player 2 Cooperate Player 1

Two-Stage Prisoners’ Dilemma Working back Player 1 Cooperate Defect Player 2 Cooperate Player 1 C C 2 R 2 R Playe Pl. 2 r 1 D D C Cooperate Defect Player 1 D C Player 1 C Pl 2 D C D R+S R+T R+P S+R R+T R+S R+P T+R Defect D Player 1 C Pl 2 C D Etc…etc C D Pl 2 D C D P+R P+S T+P P+R P+T P+S 2 P 2 P

Two-Stage Prisoners’ Dilemma Working back further Player 1 Cooperate Defect Player 2 Cooperate Player

Two-Stage Prisoners’ Dilemma Working back further Player 1 Cooperate Defect Player 2 Cooperate Player 1 C C Playe Pl. 2 r 1 D 10 21 D C Cooperate Defect Player 1 D C Player 1 C Pl 2 D C D C 21 11 10 0 11 1 10 11 21 22 11 12 Defect Player 1 D C Pl 2 D C D Pl 2 D C 11 22 12 11 0 1 D C 11 2 12 11 12 1 D

Longer Game • What is the subgame perfect outcome if Prisoners’ dilemma is repeated

Longer Game • What is the subgame perfect outcome if Prisoners’ dilemma is repeated 100 times? • Work backwards: In last round, nothing you do affects future, so you play the dominant strategy for stage game: defect. • Since last round is determined, nothing you do in next to last round affects future, so you play dominant strategy for stage game: defect • Work your way back. Only subgame perfect outcome is “Defect always”.

In a repeated game that consists of four repetitions of a stage game that

In a repeated game that consists of four repetitions of a stage game that has a unique Nash equilibrium A) There are four subgame perfect Nash equilibria B) There are 24=16 subgame perfect Nash equilibria C) There is only one subgame perfect Nash equilibrium. D) The number of subgame perfect Nash equilibria varies, depending on the details of the game.

More generally • In a subgame perfect equilibrium for a finitely repeated game where

More generally • In a subgame perfect equilibrium for a finitely repeated game where the stage game has a unique N. E, the moves in the last stage are determined for each person’s strategy. Given that the moves in the last stage don’t depend on anything that happened before, the Nash equilibrium in previous stage is uniquely determined to be the stage game equilibrium. • And so it goes…All the way back to the beginning.

Games without a last round • Two kinds of models – Games that continue

Games without a last round • Two kinds of models – Games that continue for ever – Games that end at a random, unknown time

Infinitely repeated game • • • Wouldn’t make sense to add payoffs. You would

Infinitely repeated game • • • Wouldn’t make sense to add payoffs. You would be comparing infinities. Usual trick. Discounted sums. Just like in calculating present values. We will see that cooperative outcomes can often be sustained as Nash equilibria in infinitely repeated games.

Why consider infinite games? We only have finite lives. Many games do not have

Why consider infinite games? We only have finite lives. Many games do not have known end time. Just like many human relationships. Simple example—A favorite of game theorists After each time the stage game is played there is some probability d<1 that it will be played again and probability 1 -d that play will stop. • Expected payoff “discounts” payoffs in later rounds, because game is less likely to last until then. • •

Cleaning house as a Repeated Prisoners’ Dilemma • Maybe a finite game if you

Cleaning house as a Repeated Prisoners’ Dilemma • Maybe a finite game if you have a fixed lease and don’t expect to see roommate again after lease expires. • Most relationships don’t have a known last time. • Usually some room for “residual good will. ”

In a repeated game, after each round of play, a fair coin is tossed.

In a repeated game, after each round of play, a fair coin is tossed. If it comes up heads, the game continues to another round. If it comes up tails, the game stops. What is the probability that the game is played for at least three rounds? A) B) C) D) E) 1/3 2/3 1 /4 1 /2 1/8

Calculating sums • In a repeated game, with probability d of continuation after each

Calculating sums • In a repeated game, with probability d of continuation after each round, the probability that the game is still going at round k is dk-1 • Calculate expected winnings if you receive R so long as the game continues. R+d 2 R+ d 3 R+ d 4 R + …. + =R(1+d +d 2 + d 3 + d 4 + …. + ) • What is this infinite sum?

Adding forever • The series (1+d +d 2 + d 3 + d 4

Adding forever • The series (1+d +d 2 + d 3 + d 4 + …. + ) Is known as a geometric series. When |d|<1, this series converges. That is, to say, the limit as n approaches infinity of 1+d +d 2 + d 3 + d 4 + …. + dn exists. Let S= 1+d +d 2 + d 3 + d 4 + …. + Then d. S=d +d 2 + d 3 + d 4 + …. + And S-d. S=1. So S(1 -d)=1 S=1/(1 -d).

What is the limit as n approaches infinity of 1+3/4+(3/4)2+…+(3/4)n A) B) C) D)

What is the limit as n approaches infinity of 1+3/4+(3/4)2+…+(3/4)n A) B) C) D) E) 1/(4 n) Infinity 2 4 16

Infinitely repeated prisoners’ dilemma and the “Grim Trigger Strategy” • Suppose 2 players play

Infinitely repeated prisoners’ dilemma and the “Grim Trigger Strategy” • Suppose 2 players play repeated prisoners dilemma, where the probability is d<1 that you will play another round after the end of each round. • The grim trigger strategy is to play cooperate on the first round and play cooperate on every round so long as the other doesn’t defect. • If the other defects, the grim trigger strategy plays defect on all future rounds.

When is there a symmetric SPNE where all play Grim Trigger? • Suppose that

When is there a symmetric SPNE where all play Grim Trigger? • Suppose that the other player is playing Grim Trigger. • If you play Grim Trigger as well, then you will cooperate as long as the game continues and you will receive a payoff of R. Your expected payoff from playing Grim Trigger if the other guy is playing Grim Trigger is therefore R(1+d +d 2 + d 3 + d 4 + …. + )=R/(1 -d)

What if you defect against Grim Trigger • If you defect and the other

What if you defect against Grim Trigger • If you defect and the other guy is playing Grim Trigger, you will get a payoff of T>R the first time that you defect. But after this, the other guy will always play defect. The best you can do, then is to always defect as well. • Your expected payoff from defecting is therefore T+ P(d +d 2 + d 3 + d 4 + …. + ) =T+Pd/1 -d

Cooperate vs Defect • If other guy is playing Grim trigger and nobody has

Cooperate vs Defect • If other guy is playing Grim trigger and nobody has yet defected, your expected payoff from playing cooperate is R/(1 -d) • If other guy is playing Grim trigger and nobody has yet defected, your expected payoff from playing defect is T+Pd/(1 -d) • Cooperate is. R/(1 -d) better for you if R/(1 -d)>T+Pd/(1 -d) which implies d>(T-R)/(T-P) • Example If T=10, R=5, P=2, then condition is d>5/8. • If d is too small, it pays to “take the money and run”

Other equilbria? • Grim trigger is a SPNE if d is large enough. •

Other equilbria? • Grim trigger is a SPNE if d is large enough. • Are there other SPNEs? • Yes, for example both play Always Defect is an equilibrium. • If other guy is playing Always Defect, what is your best response in any subgame? • Another is Play Defect the first 10 rounds, then play Grim Trigger.

Tit for Tat • What is both players play the following strategy in infinitely

Tit for Tat • What is both players play the following strategy in infinitely repeated P. D? • Cooperate on the first round. Then on any round do what the other guy did on the previous round. • Suppose other guy plays tit for tat. • If I play tit for tat too, what will happen?

Payoffs • If you play tit for tat when other guy is playing tit

Payoffs • If you play tit for tat when other guy is playing tit for tat, you get expected payoff of R(1+d +d 2 + d 3 + d 4 + …. + )=R/(1 -d) • Suppose instead that you choose to play “Always defect” when other guy is tit for tat. • You will get T+ P(d +d 2 + d 3 + d 4 + …. + ) =T+Pd/1 -d Same comparison as with Grim Trigger. Tit for tat is a better response to tit for tat than always defect if d>(T-R)/(T-P)

Another try • Sucker punch him and then get him to forgive you. •

Another try • Sucker punch him and then get him to forgive you. • If other guy is playing tit for tat and you play D on first round, then C ever after, you will get payoff of T on first round, S on second round, and then R for ever. Expected payoff is T+ Sd+d 2 R(1+d +d 2 + d 3 + d 4 + …. + )=T+ Sd+d 2 R/(1 -d).

Which is better? • Tit for tat and Cheat and ask forgiveness give same

Which is better? • Tit for tat and Cheat and ask forgiveness give same payoff from round 3 on. • Cheat and ask forgiveness gives T in round 1 and S in round 2. • Tit for tat give R in all rounds. • So tit for tat is better if R+d. R>T+d. S, which means d(R-S)>T-R or d>(T-R)(R-S) If T=10, R=6, and S=1, this would mean if d>4/5. But if T=10, R=5, and S=1, this would be the case only if d>5/4, which can’t happen. In this case, tit for tat could not be a Nash equilibrium.