A study of Correlated Equilibrium Polytope By Jason

A study of Correlated Equilibrium Polytope By: Jason Sorensen

The set of Correlated Equilibrium is a polytope • Polytope has two equivalent definitions: 1. )The Convex hull of a finite set of points in R^n 2. )The bounded intersection of a finite set of closed half-spaces • The correlated equilibrium constraints are • Nash Equilibrium are Correlated -> non-empty

Linear programming • Can find maximal correlated equilibrium by linear programming • Use constraints given, player i maximizing hyperplane is • Easily solved by any LP program for reasonable sized games

The Shapley game • 12 (Non-trivial) LP constraints 1 2 3 1 1, 0 0, 1 2 0, 1 1, 0 0, 0 3 0, 0 0, 1 1, 0 • 6 of them reduce to x 1 = x 2 = x 5 = x 6 = x 7 = x 9 • Reduces original 8 dimensional polytope to 9 dimensions • Unique Nash Equilibrium at xi = 1/9 • This corresponds to LP minimum for all utility hyperplanes

The Shapley polytope

Chicken or Dare? • Game matrix is: D C 0, 0 7, 2 2, 7 6, 6 • 3 Nash Equilibria at x 3 = 1, x 2 = 1, and one mixed • Each Nash Equilibirium is an LP minimum for a utility hyperplane • Can reduce polytope dimension to 3 by utilizing equality constraint

The Chicken Polytope

Theorem time! • All Nash Equilibrium lie on the boundary of the Correlated Equilibrium • Proof on the board! • … But may not by too great if polytope is not full dimensional (boundary != relative boundary)

Are CE always better than NE? • In the two cases we studied, all utility hyperplanes were minimized at an NE • Is this generally the case? • NO! • In fact, there is no way to find NE by linear systems in general

Correlated Equilibrium get Bested • Game of “Poker” (? !) 1 L R T (3, 0, 2) (0, 2, 0) B (0, 1, 0) (1, 0, 0) • • • 2 L R T (1, 0, 0) (0, 1, 0) B (0, 3, 0) (2, 0, 3) Unique Nash Equilibrium - with irrational coordinates NE does not occur at a vertex (all vertices are rational) Value of NE for player one: . 890 Value of worst case CE for player one: . 5833 Value of best case CE for player one: 1. 467 The CE Polytope is full 7 dimensional (not graphable)

Putting it all together • We can guarantee convergence to CE by natural (no-regret) learning processes • But CE may not always be better than Nash Equilibrium • Which CE do these “natural learning processes” converge to? • How long does convergence take? • Will investigate further in next 2 weeks

A Conundrum • Using learning processes to find optimal responses may result in being “bullied” by opponent (it’s always the smart ones who get picked on) • In chicken or dare, if the opponent is rational and knows you are learning, there is no reason not to always play dare • How do we decide whether to learn or bully for optimal payoffs? • Each “playing strategy” (learning or not) combated against each other in a game results in a certain payoff for each player after convergence • Model this situation as a new game, where each “move” is a learning strategy and learn the optimal strategy • Have we really learned anything?

Open Problems • Compare learning strategies to figure out optimal method • Figure out properties of general game polytopes (the number of faces) • In which situations is the polytope full dimensional? • In which situations are the NE the LP minimizing vertices for all utility hyperplanes?