Prerequisites Almost essential Game Theory Dynamic Frank Cowell

Prerequisites Almost essential Game Theory: Dynamic Frank Cowell: Microeconomics January 2007 Repeated Games MICROECONOMICS Principles and Analysis Frank Cowell

Overview. . . Repeated Games Frank Cowell: Microeconomics Basic structure Embedding the game in context… Equilibrium issues Applications

Introduction Frank Cowell: Microeconomics n n Another examination of the role of time Dynamic analysis can be difficult u u n We need an alternative approach u u n more than a few stages… …can lead to complicated analysis of equilibrium but one that preserves basic insights of dynamic games such as subgame-perfect equilibrium Build on the idea of dynamic games u u u introduce a jump from the case of comparatively few stages… …to the case of arbitrarily many

Repeated games Frank Cowell: Microeconomics n The alternative approach u u n Basic idea is simple u u n take a series of the same game embed it within a time-line structure by connecting multiple instances of an atemporal game… …model a repeated encounter between the players in the same situation of economic conflict Raises important questions u u u how does this structure differ from an atemporal model? how does the repetition of a game differ from a single play? how does it differ from a collection of unrelated games of identical structure with identical players?

History Frank Cowell: Microeconomics n n Why is the time-line different from a collection of unrelated games? The key is history u u u n History can affect the nature of the game u u n history at any point on the timeline… …is the information about actual play… …accumulated up to that point at any stage all players can know all the accumulated information strategies can be conditioned on this information History can play a role in the equilibrium u u u some outcomes that aren’t equilibria in a single encounter… …may yet be equilibria outcomes in the repeated game the game’s history is used to support such outcomes

Repeated games: Structure Frank Cowell: Microeconomics n The stage game u u u n Repeat the stage game indefinitely u u u n there’s an instance of the stage game at time 0, 1, 2, …, t, … the possible payoffs are also repeated for each t payoffs at t depends on actions in stage game at t A modified strategic environment u u n take an instant in time specify a simultaneous-move game payoffs completely specified by actions within the game all previous actions assumed as common knowledge so agents’ strategies can be conditioned on this information Modifies equilibrium behaviour and outcome?

Equilibrium Frank Cowell: Microeconomics n Simplified structure has potential advantages u u n Possibilities for equilibrium u u u n whether significant depends on nature of stage game concern nature of equilibrium new strategy combinations supportable as equilibria? long-term cooperative outcomes… …absent from a myopic analysis of a simple game Refinements of subgame perfection simplify the analysis: u u u can rule out empty threats… …and incredible promises disregard irrelevant “might-have-beens”

Overview. . . Repeated Games Frank Cowell: Microeconomics Basic structure Developing the basic concepts… Equilibrium issues Applications

Equilibrium: an approach Frank Cowell: Microeconomics n Focus on key question in repeated games: u u n Illustrate a method in an argument by example u u n how can rational players use the information from history? need to address this to characterise equilibrium Outline for the Prisoner's Dilemma game same players face same outcomes from their actions that they may choose in periods 1, 2, . . . , t, . . Prisoner's Dilemma particularly instructive given: u u its importance in microeconomics pessimistic outcome of an isolated round of the game

Prisoner’s dilemma: Reminder [L E F T ] [R IG H T ] Al f Frank Cowell: Microeconomics §Payoffs in stage game 2, 2 0, 3 3, 0 1, 1 [left] [right] Bill §If Alf plays [RIGHT] then Bill’s best response is [right]. §If Bill plays [right] then Alf’s best response is [RIGHT]. §Nash Equilibrium §Outcome that Pareto dominates NE § The highlighted NE is inefficient §Could the Pareto-efficient outcome be an equilibrium in the repeated game? §Look at the structure…

Repeated Prisoner's dilemma Frank Cowell: Microeconomics [left] (2, 2) §Stage game (t=1) Alf 1 §Stage game (t=2) follows here… §or here… [RIGHT] [LEFT] Bill [left] 2 (2, 2) 2 [left] [right] Alf [LEFT] §or here… Alf (3, 0) Bill (1, 1) [RIGHT][LEFT] Bill [right] [left] [right] (0, 3) (2, 2) (3, 0)(0, 3) [left] [right] [left] (2, 2) (0, 3) (1, 1) (3, 0) §or here… Alf (0, 3) 2 [right] [RIGHT] Bill [left] [right] (0, 3) (3, 0) (1, 1) [left] [right] (3, 0) (1, 1) [right] § Repeat this (1, 1) structure indefinitely…?

Repeated Prisoner's dilemma Frank Cowell: Microeconomics §The stage game… Alf 1 §…repeated though time [RIGHT] [LEFT] Bill [left] (2, 2) [left] [right] . . . (0, 3) . . . (3, 0) [right] . . . (1, 1) Alf t [RIGHT] [LEFT] Bill [left] (2, 2) [left] [right] . . . (0, 3) . . . (3, 0) [right] . . . (1, 1) Let's look at the detail

Repeated PD: payoffs Frank Cowell: Microeconomics n To represent possibilities in long run: u u n In the one-shot game payoffs simply represented u u u n it was enough to denote them as 0, …, 3 purely ordinal… …arbitrary monotonic changes of the payoffs have no effect Now we need a generalised notation u u n first consider payoffs available in the stage game then those available through mixtures cardinal values of utility matter we need to sum utilities, compare utility differences Evaluation of a payoff stream: u u suppose payoff to agent h in period t is uh(t) value of (uh(1), uh(2), . . . , uh(t). . . ) is given by ∞ [1 d] ∑ dt 1 uh (t) t=1 u where d is a discount factor 0 < d < 1

PD: stage game Frank Cowell: Microeconomics n A generalised notation for the stage game u u n Both socially irresponsible: u u n u they play [LEFT], [left] get (u ( *a , u*b ) where u*a > ua , u*b > ub Only Alf socially responsible: u u n they play [RIGHT], [right] get ( ua, ub) where ua > 0, ub > 0 Both socially responsible: u n consider actions and payoffs… …in each of four fundamental cases they play [LEFT], [right] get ( 0, ub) where ub > u*b Only Bill socially responsible: u u they play [RIGHT], [left] get ( ua, 0) where ua > u*a A diagrammatic view

Repeated Prisoner’s dilemma payoffs Frank Cowell: Microeconomics §Space of utility payoffs §Payoffs for Prisoner's Dilemma ub §Nash-Equilibrium payoffs §Payoffs Pareto-superior to NE _ ub §Payoffs available through mixing • §Feasible, superior points §"Efficient" outcomes U* • (u *a, u*b ) • ( _ua, u _b ) 0 • u_ a ua

Choosing a strategy: setting Frank Cowell: Microeconomics n Long-run advantage in the Pareto-efficient outcome u u n Suppose the agents recognise the advantage u u n ( *a, u*b) in each period… payoffs (u …clearly better than ( ua, ub) in each period what actions would guarantee them this? clearly they need to play [LEFT], [left] every period The problem is lack of trust: u u they cannot trust each other… …nor indeed themselves: Alf tempted to be antisocial and get payoff ua by playing [RIGHT] Bill has a similar temptation

Choosing a strategy: formulation Frank Cowell: Microeconomics n n Will a dominated outcome still be inevitable? Suppose each player adopts a strategy that 1. 2. n n Known as a trigger strategy Why the strategy is powerful u u n rewards the other party's responsible behaviour by responding with the action [left] punishes antisocial behaviour with the action [right], thus generating the minimax payoffs (ua, ub) punishment applies to every period after the one where the antisocial action occurred if punishment invoked offender is “minimaxed for ever” Look at it in detail

Repeated PD: trigger strategies Frank Cowell: Microeconomics Bill’s action in 0, …, t s. T a Alf action at t+1 Alf’s [LEFT] [left], …, [left] [RIGHT] Anything else §Take situation at t §First type of history §Response of other player to continue this history §Second type of history §Punishment response §Trigger strategies [s. Ta, s. Tb] Alf’s action in 0, …, t s. Tb [LEFT], …, [LEFT] Anything else Bill action at t+1 Bill’s [left] [right] Will it work?

Will the trigger strategy “work”? Frank Cowell: Microeconomics n n Utility gain from “misbehaving” at t: ua − u*a What is value at t of punishment from t+1 onwards? u u u n So agent chooses not to misbehave if u n ua − u*a ≤ d[u*a − ua ]/[1 − d ] But this is only going to work for specific parameters u u n Difference in utility period: u*a − ua Discounted value of this in period t+1: V : = [u*a − ua]/[1 − d ] Value of this in period t: d. V = d[u*a − ua]/[1 − d ] value of d… … relative to ua, ua and u*a What values of discount factor will allow an equilibrium?

Discounting and equilibrium Frank Cowell: Microeconomics n n n For an equilibrium condition must be satisfied for both a and b Consider the situation of a Rearranging the condition from the previous slide: u u n Simplifying this the condition must be u u n n d[u*a − ua ] ≥ [1 − d] [ ua − u*a ] d[ ua − ua ] ≥ [ ua − u*a ] d ≥ d*a where da : = [ ua − u*a ] / [ ua − ua ] A similar result must also apply to agent b Therefore we must have the condition: u u d≥d where d : = max {da , db}

Repeated PD: SPNE Frank Cowell: Microeconomics n n Assuming d ≥ d , take the strategies [s. Ta , s. Tb ] prescribed by the Table If there were antisocial behaviour at t consider the subgame that would then start at t+1 u u n But if [LEFT], [left] has been played in every period up till t: u u u n Alf could not increase his payoff by switching from [RIGHT] to [LEFT], given that Bill is playing [left] a similar remark applies to Bill so strategies imply a NE for this subgame likewise for any subgame starting after t+1. Alf would not wish to switch to [RIGHT] a similar remark applies to Bill again we have a NE So, if d is large enough, [s. T a, s. T b] is a Subgame-Perfect Equilibrium u yields the payoffs ( u*a, u*b) in every period

Folk Theorem Frank Cowell: Microeconomics n The outcome of the repeated PD is instructive u u n Strictly speaking a class of results u u n illustrates an important result …the Folk Theorem finite/infinite games different types of equilibrium concepts A standard version of the Theorem: u u In a two-person infinitely repeated game: if the discount factor is sufficiently close to 1 any combination of actions observed in any finite number of stages… …is the outcome of a subgame-perfect equilibrium

Assessment Frank Cowell: Microeconomics n The Folk Theorem central to repeated games u u n n Clearly has considerable attraction Put its significance in context u u n perhaps better described as Folk Theorems a class of results makes relatively modest claims gives a possibility result Only seen one example of the Folk Theorem u u let’s apply it… …to well known oligopoly examples

Overview. . . Repeated Games Frank Cowell: Microeconomics Basic structure Some well-known examples… Equilibrium issues Applications

Cournot competition: repeated Frank Cowell: Microeconomics n Start by reinterpreting PD as Cournot duopoly u u u n Possible actions and outcomes in the stage game: u u u n [HIGH], [high]: both firms get Cournot-Nash payoff PC > 0 [LOW], [low]: both firms get joint-profit maximising payoff PJ > PC [HIGH], [low]: payoffs are ( P, 0) where P > PJ Folk theorem: get SPNE with payoffs (PJ , PJ ) if d is large enough u n two identical firms can each choose one of two levels of output – [high] or [low] can firms sustain a low-output (i. e. high-profit) equilibrium? Critical value for the discount factor d is P − PJ d = ────── P − PC But we should say more u Let’s review the standard Cournot diagram

Cournot stage game Frank Cowell: Microeconomics q 2 §Firm 1’s Iso-profit curves §Firm 2’s Iso-profit curves §Firm 1’s reaction function §Firm 2’s reaction function q 2 l c 1(·) §Cournot-Nash equilibrium §Outputs with higher profits for both firms §Joint profit-maximising solution §Output that forces other firm’s profit to 0 (q. C 1, q. C 2) l c 2(·) l (q 1 J, q. J 2) 0 l q 1

Repeated Cournot game: Punishment Frank Cowell: Microeconomics n Standard Cournot model is richer than simple PD: u u n Minimax profit level for firm 1 in a Cournot duopoly u u n is zero, not the NE outcome PC arises where firm 2 sets output to q 2 such that 1 makes no profit Imagine a deviation by firm 1 at time t u n action space for PD stage game just has the two output levels continuum of output levels introduces further possibilities raises q 1 above joint profit-max level Would minimax be used as punishment from t+1 to ∞? u u clearly (0, q 2) is not on firm 2's reaction function so cannot be best response by firm 2 to an action by firm 1 so it cannot belong to the NE of the subgame everlasting minimax punishment is not credible in this case

Repeated Cournot game: Payoffs Frank Cowell: Microeconomics §Space of profits for the two firms P 2 P §Cournot-Nash outcome • §Joint-profit maximisation §Minimax outcomes §Payoffs available in repeated game (PJ, PJ) (PC, PC) 0 • P P 1 Now review Bertrand compewtition

Bertrand stage game Frank Cowell: Microeconomics p 2 §Marginal cost pricing §Monopoly pricing §Firm 1’s reaction function §Firm 2’s reaction function §Nash equilibrium p. Ml cl l l c l p. M p 1

Bertrand competition: repeated Frank Cowell: Microeconomics n NE of the stage game: u u n NE outcome is the minimax outcome u u n minimax outcome is implementable as a Nash equilibrium… … in all the subgames following a defection from cooperation In repeated Bertrand competition u u n set price equal to marginal cost c results in zero profits firms set p. M if acting “cooperatively” split profits between them if one firm deviates from this… …others then set price to c Repeated Bertrand: result u u can enforce joint profit maximisation through trigger strategy… …provided discount factor is large enough

Repeated Bertrand game: Payoffs Frank Cowell: Microeconomics §Space of profits for the two firms P 2 PM §Bertrand-Nash outcome • §Firm 1 as a monopoly §Firm 2 as a monopoly §Payoffs available in repeated game 0 • P P 1 M

Repeated games: summary Frank Cowell: Microeconomics n New concepts: u u n Stage game History The Folk Theorem Trigger strategy What next? u Games under uncertainty