Game Theory Robin Burke GAM 224 Spring 2004

Outline Admin ¢ Game Theory ¢ Utility theory l Zero-sum and non-zero sum games l Decision Trees l Degenerate strategies l

Admin Due Wed l Homework #3 ¢ Due Next Week l Rule Analysis ¢ Reaction papers l Grades available ¢

Game Theory ¢ ¢ ¢ A branch of economics Studies rational choice in a adversarial environment Assumptions l l rational actors complete knowledge • in its classic formulation l l known probabilities of outcomes known utility functions

Utility Theory ¢ Utility theory a single scale l value with each outcome l ¢ Different actors may have different utility valuations l but all have the same scale l

Expected Utility ¢ Expected utility what is the likely outcome l of a set of outcomes l each with different utility values l ¢ Example l Bet • $5 if a player rolls 7 or 11, lose $2 otherwise l Any takers?

How to evaluate ¢ Expected Utility l for each outcome • reward * probability l ¢ Meaning l ¢ (1/6) * 5 + (1/18) * 5 + (7/9) (-2) = -2/9 If you made this bet 1000 times, you would probably end up $222 poorer. Doesn't say anything about how a given trial will end up l Probability says nothing about the single case

Game Theory Examine strategies based on expected utility ¢ The idea ¢ l a rational player will choose the strategy with the best expected utility

Example ¢ ¢ ¢ Non-probabilistic Cake slicing Two players l l cutter chooser Cutter's Utility Choose bigger piece smaller piece Cut cake evenly ½ - a bit ½ + a bit Cut unevenly Small piece Big piece

Rationality ¢ Choose bigger piece smaller piece Cut cake evenly (-1, +1) (+1, -1) Cut unevenly (-10, +10) (+10, -10) Rationality l l ¢ Cutter's Utility each player will take highest utility option taking into account the other player's likely behavior In example l if cutter cuts unevenly • he might like to end up in the lower right • but the other player would never do that • -10 l if the current cuts evenly, • he will end up in the upper left • -1 • this is a stable outcome • neither player has an incentive to deviate

Zero-sum ¢ Note l for every outcome • the total utility for all players is zero ¢ Zero-sum game l l something gained by one player is lost by another zero-sum games are guaranteed to have a winning strategy • a correct way to play the game ¢ Makes the game not very interesting to play l to study, maybe

Non-zero sum ¢ A game in which there are nonsymmetric outcomes l ¢ better or worse for both players Classic example l Prisoner's Dilemma Hold Out Confess Hold Out [-1, -1] [-3, 0] Confess [-5, -5] [0, -3]

Degenerate Strategy ¢ A winning strategy is also called l ¢ Because l l ¢ a degenerate strategy it means the player doesn't have to think there is a "right" way to play Problem l l game stops presenting a challenge players will find degenerate strategies if they exist

Nash Equilibrium ¢ Sometimes there is a "best" solution l ¢ A Nash equilibrium is a strategy l l ¢ Even when there is no dominant one in which no player has an incentive to deviate because to do so gives the other an advantage Creator l l l John Nash Jr "A Beautiful Mind" Nobel Prize 1994

Classic Examples ¢ Car Dealers l l Why are they always next to each other? Why aren't they spaced equally around town? • Optimal in the sense of not drawing customers to the competition ¢ Equilibrium l l because to move away from the competitor is to cede some customers to it

Prisoner's Dilemma ¢ Nash Equilibrium l ¢ Because l ¢ Confess in each situation, the prisoner can improve his outcome by confessing Solution l l l iteration communication commitment

Rock-Paper-Scissors Player 2 Player 1 Rock Paper Scissors Rock [0, 0] [-1, +1] [+1, -1] Paper [+1, -1] [0, 0] [-1, +1] Scissors [-1, +1] [+1, -1] [0, 0]

No dominant strategy ¢ Meaning l there is no single preferred option • for either player ¢ Best strategy (single iteration) l choose randomly l "mixed strategy" l

Mixed Strategy ¢ ¢ Important goal in game design Player should feel l all of the options are worth using l none are dominated by any others Rock-Paper-Scissors dynamic l is often used to achieve this Example l Warcraft II • • Archers > Knights > Footmen > Archers must have a mixed army

Mixed Strategy 2 ¢ ¢ Other ways to achieve mixed strategy Ignorance l If the player can't determine the dominance of a strategy • a mixed approach will be used • (but players will figure it out!) ¢ Cost l Dominance is reduced • if the cost to exercise the option is increased • or cost to acquire it ¢ Rarity l Mixture is required • if the dominant strategy can only be used periodically or occasionally ¢ Payoff/Probability Environment l Mixture is required • if the probabilities or payoffs change throughout the game

Mixed Strategy 3 ¢ In a competitive setting l l ¢ mixed strategy may be called for even when there is a dominant strategy Example l l l Football third down / short yardage highest utility option • running play • best chance of success • lowest cost of failure ¢ But l if your opponent assumes this • defenses adjust l increasing the payoff of a long pass

Degeneracies Are not always obvious ¢ May be contingent on game state ¢

Example ¢ Liar's Dice l l roll the dice in a cup state the "poker hand" you have rolled stated hand must be higher than the opponent's previous roll opponent can either • accept the roll, and take his turn, or • say "Liar", and look at the dice l if the description is correct • opponent pays $1 l if the description is a lie • player pays $1

Lie or Not Lie ¢ Make outcome chart for next player l assume the roll is not good enough l ¢ Roller l ¢ lie or not lie Next player l accept or doubt

Expectation ¢ Knowledge the opponent knows more than just this l the opponent knows the previous roll that the player must beat l • probability of lying

Note ¢ The opponent will never lie about a better roll l ¢ Outcome cannot be improved by doing so The opponent cannot tell the truth about a worse roll l Illegal under the rules

Expected Utility ¢ ¢ What is the expected utility of the doubting strategy? l P(worse) - P(better) When P(worse) is greater than 0. 5 l doubt Probabilities l pair or better: 95% l 2 pair or better: 71% l 3 of a kind or better: 25% So start to doubt somewhere in the middle of the twopair range l maybe 4 s-over-1 s

BUT ¢ There is something we are ignoring

Repeated Interactions Roll 1 doubt Truth Lie doubt accept Win Lose Roll 2 doubt Lie doubt Truth accept doubt Roll 1 Truth Lie accept Roll 2

Decision Tree ¢ ¢ ¢ Examines game interactions over time Each node l Is a unique game state Player choices l create branches Leaves l end of game (win/lose) Important concept for design l usually at abstract level l question • can the player get stuck? ¢ Example l tic-tac-toe

Future Cost ¢ There is a cost to "accept" l l ¢ To compare doubting and accepting l ¢ I may be incurring some future cost because I may get caught lying we have to look at the possible futures of the game In any case l l the game becomes degenerate what is the effect of adding a cost to "accept"?

Reducing degeneracy Come up with a rule for reducing degeneracy in this game ¢ Ideally, both options (accept, doubt) would continue to be valid ¢ l no matter what the state of the game is

Wednesday ¢ Analysis Case Study l Final Fantasy Tactics Advance