Multiagent Systems Game Theory Manfred Huber 2018 1

Multiagent Systems Game Theory © Manfred Huber 2018 1

Non-Cooperative Game Theory n Game Theory is the study of the interaction and decision making of rational, selfinterested agents n Each agent has its own interests n While these are mostly distinct, non-cooperative does not preclude that agents have common interests n Agents are rational and only pursue their interests n The individual agent is the basic unit n © Manfred Huber 2018 Collaborative or coalitional game theory model teams as basic units 2

Game Theory n TCP backoff example n n Agents can use a correct or an incorrect TCP backoff mechanism n Correct mechanism backs off after package collisions n Incorrect mechanism does not increase backup over time Collisions of packets from multiple agents can lead to delays due to retries n n n © Manfred Huber 2018 Correct implementation leads to an expected delay of 1 Incorrect implementation immediately retries, causing a correct implementation to back off further, leading to a delay of 4 Two incorrect implementations clash multiple times, leading to an expected delay of 3 3

Game Theory n The interaction and decision process of the agents is modeled as a game with separate utility functions for each agent n What action should each player take ? n Will both agents behave the same ? n Do the precise numbers influence behavior ? n Would communication change behavior ? n n Should actions change when a game is played repeatedly ? Is it important that agents are assumed to be rational ? © Manfred Huber 2018 4

Single Shot Games n Single shot (or single stage) games are games where multiple agents have to make a single action choice. n n n Past experiences have no influence on the current game and the agents’ decisions Utility is equal to the expected reward Finite n-person game n N is a finite set of n agents (players) n is the joint action set of the agents n is the joint utility function for the agent n © Manfred Huber 2018 5

Matrix Games n A finite single stage game can be written in the form of an n-dimensional matrix (called “normal form”) n n n Each dimension enumerates the actions of one agent Entries in the matrix are n-tuples of utility values for each agent for outcome of the given action choices The TCP backoff example Correct Incorrect © Manfred Huber 2018 Correct -1, -1 -4, 0 Incorrect 0, -4 -3, -3 6

Matrix Games n The prisoners’ dilemma Cooperate Stay silent © Manfred Huber 2018 Cooperate -5, -5 -1, -10 Stay silent -10, -1 -3, -3 7

Canonical Games n Pure competition games are 2 -player constant sum games n n n Every utility gain for one agent is an equal utility loss for the other agents Only one utility function is needed Example: Matching Pennies © Manfred Huber 2018 Heads Tails Heads 1, -1 -1, 1 Tails -1, 1 1, -1 8

Pure competition Games n Example: Rock, Paper, Scissors Rock Paper Scissors Rock 0, 0 -1, 1 1, -1 Paper 1, -1 0, 0 -1, 1 1, -1 0, 0 Scissors -1, 1 © Manfred Huber 2018 9

Canonical Games n Coordination games are games where players have exactly the same interests n All players obtain the same utilities n Only one utility function is needed n Note: this is still non-cooperative since each agent makes its own decision n Example: Driving on a side of the road © Manfred Huber 2018 Left Right Left 1, 1 -1, -1 Right -1, -1 1, 1 10

Canonical Games n n General games contain elements of cooperation and competition Example: Battle of the sexes © Manfred Huber 2018 Opera Football Opera 3, 1 0, 0 Football 0, 0 1, 3 11

Decision-Making: Analyzing Games n n “Better” outcomes/strategies have to be defined n Utilities for different agents can not be compared n All agents are equally important Pareto dominance n An outcome o Pareto-dominates an outcome o’ if for every agent outcome o is at least as good as outcome o’ © Manfred Huber 2018 12

Pareto Optimality n An outcome/strategy is Pareto optimal if it is not Pareto dominated by any other outcome n n n Every game has at least one Pareto optimal strategy Every game has at least one Pareto optimal pure strategy Games can have multiple Pareto optimal strategy © Manfred Huber 2018 13

Pareto Optimality n n Pareto Optimality allows to identify better strategies from an outside point of view. Examples: Heads Tails Heads 1, -1 -1, 1 Tails -1, 1 1, -1 Opera Football Opera 3, 1 0, 0 Football 0, 0 1, 3 © Manfred Huber 2018 Left Right Left 1, 1 -1, -1 Right -1, -1 1, 1 Cooperate Stay silent Cooperate -5, -5 -1, -10 Stay silent -10, -1 -3, -3 14

Decision-Making: Best Response n If an agent knows the strategies of all other agents it can easily pick its own action n n Best response: When the strategies of other agents are not known we have to look for “stable” strategies n A strategy is a (pure strategy) Nash equilibrium if for every agent its action is a best response to the actions of the other agents © Manfred Huber 2018 15

Nash Equilibrium n A Nash Equilibrium is a stable strategy n n n Any agent deviating from it will not increase its utility An agent that is aware of the other agents’ strategies can not increase its utility using this knowledge (the equilibrium is its best strategy) In an equilibrium it is possible to determine the other agents’ strategies based on the assumption of rationality © Manfred Huber 2018 16

Nash Equilibrium n n Nash equilibria allow agents to identify stable strategies that lead to “best” outcomes. Examples: Heads Tails Heads 1, -1 -1, 1 Tails -1, 1 1, -1 Opera Football Opera 3, 1 0, 0 Football 0, 0 1, 3 © Manfred Huber 2018 Left Right Left 1, 1 -1, -1 Right -1, -1 1, 1 Cooperate Stay silent Cooperate -5, -5 -1, -10 Stay silent -10, -1 -3, -3 17

Nash Equilibrium n Many games do not have pure strategy equilibria n n In competitive settings it is often a bad idea to always do the same If other agents know your actions they can use this knowledge to utilize your weaknesses © Manfred Huber 2018 18

Mixed Strategies n Agents can follow different strategies n n n Pure strategies are strategies where agents select actions deterministically Mixed strategies are strategies where actions are taken according to a probability distribution Strategy profiles are the joint strategies of all the agents © Manfred Huber 2018 19

Utility of Mixed Strategies n The utility of a mixed strategy profile can be computed as the expected value of the outcome lottery © Manfred Huber 2018 20

Best Response and Nash Equilibrium n If an agent knows the strategies of all other agents it can pick its own (mixed) strategy n n Best response: When the strategies of other agents are not known we have to look for “stable” strategies n A strategy profile is a Nash equilibrium if for every agent its strategy is a best response to the strategies of the other agents © Manfred Huber 2018 21

Nash Equilibrium n n A Strict Nash equilibrium is a Nash equilibrium where for each agent the strategy is a strictly better (higher utility) response to the other agents’ strategies than any other strategy A Weak Nash equilibrium is any Nash equilibrium that is not a Strict Nash equilibrium n All mixed strategy equilibria are Weak © Manfred Huber 2018 22

Nash Equilibrium n Nash (1951): Every finite game has at least one Nash equilibrium n Computing Nash equilibria can be complex n It becomes easier if the support of the strategy (i. e. the set of actions with probabilities greater than 0) is known Heads Tails Heads 1, -1 -1, 1 Tails -1, 1 1, -1 © Manfred Huber 2018 Opera Football Opera 3, 1 0, 0 Football 0, 0 1, 3 23

Solution Approaches n To address decision making in matrix games we need algorithms that can determine Nash equilibria n Maxmin and Minmax for constant sum games n Linear programming for (zero-sum) matrix games n n Linear Complimentarity Problem (LPC) for general 2 -player games Lemke-Howson algorithm © Manfred Huber 2018 24