Evolutionary Game Theory Amit Bahl CIS 620 Outline

Evolutionary Game Theory Amit Bahl CIS 620

Outline z. EGT versus CGT z. Evolutionary Stable Strategies Concepts and Examples z. Replicator Dynamics Concepts and Examples z. Overview of 2 papers Selection methods, finite populations

EGT v. Conventional Game Theory z. Models used to study interactive decision making. z. Equilibrium is still at heart of the model. z. Key difference is in the notion of rationality of agents.

Agent Rationality z. In GT, one assumes that agents are perfectly rational. z. In EGT, trial and error process gives strategies that can be selected for by some force (evolution - biological, cultural, etc…). z. This lack of rationality is the point of departure between EGT and GT.

Evolution z. When in biological sense, natural selection is mode of evolution. z. Strategies that increase Darwinian fitness are preferable. z. Frequency dependent selection.

Evolutionary Game Theory (EGT) Has origins in work of R. A. Fisher [The Genetic Theory of Natural Selection (1930)]. • Fisher studied why sex ratio is approximately equal in many species. • Maynard Smith and Price introduce concept of an ESS [The Logic of Animal Conflict (1973)]. • Taylor, Zeeman, Jonker (1978 -1979) provide continuous dynamics for EGT (replicator dynamics).

ESS Approach z. ESS = Nash Equilibrium + Stability Condition z. Notion of stability applies only to isolated bursts of mutations. z. Selection will tend to lead to an ESS, once at an ESS selection keeps us there.

ESS - Definition • Consider a 2 player symmetric game with ESS given by I with payoff matrix E. • Let p be a small percentage of population playing mutant strategy J I. • Fitness given by W(I) = W 0 + (1 -p)E(I, I) + p. E(I, J) W(J) = W 0 + (1 -p)E(J, I) + p. E(J, J) • Require that W(I) > W(J)

ESS - Definition z. Standard Definition for ESS (Maynard Smith). z. I is an ESS if for all J I, E(I, I) E(J, I) and E(I, I) = E(J, I) E(I, J) > E(J, J) where E is the payoff function.

ESS - Definition Assumptions: 1) Pairwise, symmetric contests 2) Asexual inheritance 3) Infinite population 4) Complete mixing

ESS - Existence z. Let G be a two-payer symmetric game with 2 pure strategies such that E(s 1, s 1) E(s 2, s 1) AND E(s 1, s 2) E(s 2, s 2) then G has an ESS.

ESS Existence z. If a > c, then s 1 is ESS. z. If d > b, then s 2 is ESS. z. Otherwise, ESS given by playing s 1 with probability equal to (b-d)/[(b-d)+(a-c)].

ESS - Example 1 z. Consider the Hawk-Dove game with payoff matrix z. Nash equilibrium given by (7/12, 5/12). z. This is also an ESS.

ESS - Example 1 z. Bishop-Cannings Theorem: If I is a mixed ESS with support a, b, c, …, then E(a, I) = E(b, I) = … = E(I, I). z. At a stable polymorphic state, the fitness of Hawks and Doves must be the same. z. W(H) = W(D) => The ESS given is a stable polymorphism.

Stable Polymorphic State

ESS - Example 2 z. Consider the Rock-Scissors-Paper z. Payoff matrix is given by R R -e S -1 P 1 Game. S P 1 -1 -e z. Then I = (1/3, 1/3) is an ESS but stable polymorphic population 1/3 R, 1/3 P, 1/3 S is not stable.

ESS - Example 3 z. Payoff matrix : z. Then I = (1/3, 1/3) is the unique NE, but not an ESS since E(I, s 1)=E(s 1, s 1)= 1.

Sex Ratios z. Recall Fisher’s analysis of the sex ratio. z. Why are there approximately equal numbers of males and females in a population? z. Greatest production of offspring would be achieved if there were many times more females than males.

Sex Ratios z. Let sex ratio be s males and (1 -s) females. z. W(s, s’) = fitness of playing s in population of s’ z. Fitness is the number of grandchildren z. W(s, s’) = N 2[(1 -s) + s(1 -s’)/s’] W(s’, s’) = 2 N 2(1 -s’) z. Need s* s. t. s W(s*, s*) W(s, s*)

Dynamics Approach z. Aims to study actual evolutionary process. z. One Approach is Replicator Dynamics. z. Replicator dynamics are a set of deterministic difference or differential equations.

RD - Example 1 z. Assumptions: Discrete time model, nonoverlapping generations. zxi(t) = proportion playing i at time t z (i, x(t)) = E(number of replacement for agent playing i at time t) z j {xj(t) (j, x(t))} = v(x(t)) zxi(t+1) = [xi(t) (i, x(t))]/ v(x(t))

RD - Example 1 z. Assumptions: Discrete time model, nonoverlapping generations. zxi(t+1) - xi(t) = xi(t) [ (i, x(t)) - v(x(t))] v(x(t))

RD - Example 2 z. Assumptions : overlapping generations, discrete time model. z. In time period of length , let fraction give birth to agents also playing same strategy. z j xj(t)[1 + (j, x(t))] = v(x(t)) zxi(t+ ) = xi(t)[1 + (i, x(t))] v(x(t))

RD - Example 2 z. Assumptions : overlapping generations, discrete time model. zxi(t+ ) - xi(t)= xi(t)[ (i, x(t)) - v(x(t))] 1+ v(x(t))

RD - Example 3 z. Assumptions: Continuous time model, overlapping generations. z. Let 0, then dxi /dt = xi(t)[ (i, x(t)) - v(x(t))]

Stability z. Let x(x 0, t): S X R S be the unique solution to the replicator dynamic. z. A state x S is stationary if dx/dt = 0. z. A state x is stable if it is stationary and for every neighborhood V of x, there exists a U V s. t. x 0 U, t x(x 0, t) V.

Propostions for RD z. If (x, x) is a NE, then x is a stationary state of the RD. y dxi /dt = xi(t)[ (i, x(t)) - v(x(t))] z. What about the converse? y. Consider population of all doves.

Propostions for RD z. If x is a stable state of the RD, then (x, x) is a NE. y. Consider any perturbation that introduces a better reply. z. What about the converse? Consider:

Stronger notion of Stability z. A state x is asymptotically stable if it is stable and there exists a neighborhood V of x s. t. x 0 V, limt x(x 0, t) = x.

ESS and RD z. In general, every ESS is asymptotically stable. z. What about the converse?

ESS and RD z. Consider the following game: z. Unique NE given by x* = (1/3, 1/3). z. If x = (0, 1/2), then E(x, x*)=E(x*, x*)=2/3 but E(x, x)=5/4 > 7/6=E(x*, x).

ESS and RD In 2 X 2 games, x is an ESS if and only if x is asymptotically stable.

A Game-Theoretic Investigation of Selection Methods Used in Evolutionary Algorithms Ficici, Melnik, Pollack

Selection Methods z. How do common selection methods used in evolutionary algorithms function in EGT? z. Dynamics and fixed points of the game.

Selection function zxi(t+1) = S(F(xi(t)), xi(t)) where S is the selection function, F is the fitness function, and xi(t) is the proportion of population playing i at time t.

Fitness Dependent Selection f’ = (p X f)/(p • f) {x(x 0, t): t R} = orbit passing through x 0.

Truncation Selection 1) Sort by fitness 2) Replace k% of lowest by k% of highest

Truncation Selection z. Consider the Hawk-Dove game with ESS given by (7/12 H, 5/12 D) If. 5 < x. H(t) < 7/12, then x. H(t+1) = 1.

Truncation Selection Map Diagram:

( , )-ES Selection z Given a population of offspring, the best are chosen to parent the next generation. z More extreme than truncation selection.

Linear Rank Selection z. Agents sorted according to fitness. z. Assigned new fitness values according to their rank. z. Causes fitness to change linearly with rank. z. Causes cycles around ESS.

Linear Rank Selection Map Diagram:

Boltzman Selection z. Inspired by simulated annealing. z. Selection pressure slowly increased over time to focus search. z. In some cases, BS is able to retain the dynamics and equilibria in EGT.

Boltzman Selection Map Diagram:

Effects of Finite Populations on Evolutionary Stable Strategies. Ficici, Pollack

Finite Populations z. Effects of finite population on EGT. z. Begin at ESS (7/12, 5/12) and test n=60, 120, 300, 600, and 900 for 2000 generations. z 100 replicates of each experiment.

Finite Populations z. Results:

Convergence z. For a n player name, consider the MC with states given by #of hawks. z. Define transition matrix P. zb t = b 0 P t z. E(x. H(t)) = (1/n) ni=0 b. Ht * i zlimt E(x. H(t)) = b H z. Estimate | E(x. H(t)) - b H |

Convergence Simulation z. Results: