The Computational Complexity of Finding a Nash Equilibrium

The Computational Complexity of Finding a Nash Equilibrium Edith Elkind, U. of Warwick

Based On… • Reducibility Among Equilibrium Problems (Goldberg, Papadimitriou): Aug 2005 • The Complexity of Computing a Nash Equilibrium (Goldberg, Daskalakis, Papadimitriou): Sep 2005 • 3 -NASH is PPAD-Complete (Chen, Deng): Nov 2005 • Three-Player Games Are Hard (Daskalakis, Papadimitriou): Nov 2005 • Settling the Complexity of 2 -Player Nash. Equilibrium (Chen, Deng): Dec 2005

Normal Form Games • finite set of players {1, …, n} • each player has k actions (pure strategies): 1, …, k • payoffs of the ith player: Pi: {1, …, k}n → R 0 Row player: 1 0 2 0 1 0 Column player: 1 0 1 0 3

Nash Equilibrium • Nash equilibrium: a strategy profile such that noone wants to deviate given other players’ strategies, i. e. , each player’s strategy is a best response to others’ strategies: – (0, 0) and (1, 1) are both NE 0 Row player: 1 0 2 0 1 0 Column player: 1 0 1 0 3

Pure vs. Mixed Strategies • NE in pure strategies may not exist! – “matching pennies” • Mixed strategy: a probability distribution over actions – 50% tail, 50% head H Row player: T H 1 -1 T -1 1 Column player: H T H -1 1 T 1 -1

Existence of NE • Theorem (Nash 1951): any game in normal form has an equilibrium in mixed strategies $1 000 question: how to find one?

Finding mixed NE in 2 x 2 Games Suppose C plays 1 w. p. c EP(R) from playing 0: (1 -c)*2 EP(R) from playing 1: c*1 (1 -c)*2 = c iff c = 2/3 Suppose R plays 1 w. p. r EP(C) from playing 0: (1 -r)*1 EP(C) from playing 1: r*3 1 -r = 3 r iff r = ¼ NE: r=1/4, c=2/3 0 Row player: 1 0 2 0 1 0 Column player: 1 0 1 0 3

2 players, k actions • Representation: two k x k matrices • Checking for pure NE: easy – at most k 2 of them • Checking for mixed NE: – all straightforward methods are exptime – Lemke-Howson algorithm is exptime, too (previous talk) • For 2 players all NE are rational – but not for 3 and more players…

n players, 2 actions • Representation: payoffs to each player for every action profile (vector in {0, 1}n): n 2 n numbers • graphical games: – players are associated with the vertices of a graph; – each player’s payoff depends on his own action and actions of his neighbors – n players, max degree d => n 2 d+1 numbers W T U V W’s payoffs (16 cases): t=0, u=0, v=0, w=0: 12 t=1, u=0, v=0, w=0: 31 …. t=1, u=1, v=1, w=1: -6

Algorithms for NE in Graphical Games • Bounded-degree trees: – Exp-time algorithm/poly-time approximation algorithm to find all NE (Kearns, Littmann, Singh, UAI 2001) – ? ? ? poly-time algorithm to find a single NE (Kearns, Littmann, Singh, NIPS’ 2001): • shown to be incorrect in E. , Goldberg, ACM EC’ 06 • Graphs of max degree 2: – poly-time algorithm (EGG’ 06)

Is Finding NE NP-hard? • Reminder: a problem P is NP-hard if you can reduce 3 -SAT to it: – “yes”-instance 3 -SAT → “yes”-instance of P – “no”-instance 3 -SAT → “no”-instance of P • Problem: each instance of NASH is a “yes”-instance! – every game has a NE – need complexity theory for search problems • Side note: pure Nash for n players, NE of total value > K are NP-hard

Reducibility Among Search Problems S: X Y f T: X’ g Y’ • S associates x in X with a solution set S(x) • Total search problem: for any x, S(x) is not empty • S is reducible to T if: – f, g easy to compute – g(T(f(x))) is in S(x) If T is easy, so is S

Equivalences: GP’ 05 r-player game G NE of G f g deg 3 graphical game G’ NE of G’ deg d graphical game G NE of G f d 2 -player game G’ g NE of G’

d-Graphical Game GG → d 2 -Player Game G • Color the graph of GG: d(u, v) ≤ 2 color(u) ≠ color(v) • Each color is a player of G • RED chooses a red vertex in GG and an action for that vertex in GG • payoff=payoff 1+payoff 2 – payoff 1: BLUE tries to guess which vertex RED chose; RED pays a penalty if BLUE guesses correctly – payoff 2: if all neighbors of a chosen vertex are also chosen, it gets same payoff as in GG, else 0

r-Player Game G → 3 -Graphical Game GG • • • Si: space of pure strategies of player i S- i = S 1 * … Si-1*Si+1 *. . * Sr xij: the probability that ith player uses jth strategy xs: x 1 s 1 * x 2 s 2 … * xrsr (for s in S-i) uijs: utility of the ith player when he plays j and others play according to s NE: 0 ≤ xij ≤ 1 Sj xij =1 Ss in S uijsxs > Ss in S uij’sxs implies xij’ = 0 -p -p

r-Player Game G → 3 -Graphical Game GG • Vertex Vij for any pair (player=i, action=j) • Want: Pr[Vij plays 1] = Pr [i plays j in G]=xij • Idea: graphical games can do math! – Enforce constraints from the previous slide… v 1 v 2 v 3 Need gadgets for +, *, c, =, min, max, … u Set payoffs to u, v 3 so that p[v 3]=p[v 1] * p[v 2]

Equivalences: GP’ 05 r-player game G NE of G f g deg 3 graphical game G’ NE of G’ deg d graphical game G NE of G f d 2 -player game G’ g NE of G’

Combining Reductions: GP’ 05 r-player game G 4 X 9 -player game G’ NE of G f g NE of G’ Finding NE in a 4 -player game is as hard as finding NE in a r-player game for any constant r

Completeness Results? • Can we prove that any total search problem is reducible to r-NASH? • Not really: the class T of all total search problems is a semantic class – not known how to find complete problems for these • Want to pick a large subclass S of T s. t. – S includes some natural problems – there are problems that are complete for S – in particular, r-NASH is complete for S

END OF THE LINE • Input: Boolean circuits S (Successor), P (Predecessor): – n inputs, n outputs – S(0 n) ≠ 0 n, P(0 n) = 0 n • Output: x ≠ 0 n s. t. – S(P(x)) ≠ x or P(S(x)) ≠ x Intuition: G=(V, E): – V = S n; – E = {(x, y) | y=S(x), x=P(y)} 00000 11001 01011

PPAD • PPAD: Polynomial Parity Argument, Directed version • PPAD is the class of all search problems that are reducible to END OF THE LINE search problem solution f circuits S, T g “end of the line”

r-NASH is in PPAD • Proof on Nash’s theorem: – existence of NE reduces to Brouwer’s fixpoint theorem – Brouwer’s fixpoint theorem reduces to Sperner’s lemma – Sperner’s lemma is proven by a parity argument (similar to END OF THE LINE) • Reduction of r-NASH to END OF THE LINE can be extracted from these proofs (Papadimitriou 94)

Brouwer’s Fixpoint Theorem • Brouwer’s Theorem: Any continuous mapping from the simplex to itself has a fixpoint. • Nash Brouwer proof sketch: – set of all strategy profiles → simplex – mapping: (s 1, …, sn) → (s 1+d 1, …, sn+dn), where di is a shift in the direction of best response to (s 1, …, si-1, si+1, …, sn) – NE is a point where noone wants to deviate, i. e. , a fixpoint

Sperner’s Lemma • Proper coloring: B – vertices on BC are not blue – vertices on AC are not green – vertices on AB are not yellow A • Sperner’s Lemma: there exists a trichromatic triangle • Brouwer’s theorem Sperner’s Lemma: – x is blue if the grad(F) at x points away from A, etc. – trichromatic triangle “has no direction” – repeat at increased resolution… C

Opposite Direction: 3 D-BROUWER • Input: – 3 D unit cube divided into 23 n cubelets – cijk is the center of Kijk – f(cijk)=cijk+dijk, dijk is in {d 0, d 1, d 2, d 3}, where • d 1=(a, 0, 0), d 2=(0, a, 0) • d 3=(0, 0, a), d 0=(-a, -a) – circuit C: {0, 1} 3 n → {0, 1, 2, 3} selects dijk • Output: – a panchromatic cubelet, i. e. , one that has all of d 0, d 1, d 2, d 3 among its 8 neighbors

3 D-BROUWER is PPAD-complete • Papadimitriou (1994) shows that a more complicated version of 3 D-BROUWER is PPAD-complete • This version was proven hard in DGP’ 05 • Reduction from END OF THE LINE – embed the line L into 3 d cube – “protect” L from color 0 using three other colors – color the rest of inner cubelets with 0

r-NASH vs 3 D BROUWER • Existence of NE follows from Brouwer’s fixpoint theorem • NE are special cases of Brouwer’s fixpoints – just how special? • Can any fixpoint be represented as a NE of a game? • DGP’ 05: YES! 4 -NASH is PPAD complete • Proof: – 4 -NASH deg 3 Graphical Nash – graphical games can compute fixpoints

4 -NASH to 3 -NASH • Daskalakis, Papadimitriou: modify arithmetic gadgets so that the graph is 3 -colorable • Chen, Deng: same gadgets, but allow for small error

2 -NASH • Chen, Deng: – avoid graphical games – reduce directly from 3 D-BROUWER to -NASH using arithmetic gadgets similar to graphical game gadgets • Game over? 2

Graphical Games: Open Problems • Degree: – deg 3 PPAD-complete (DGP’ 05 b) – deg 2 polynomial time solvable (EGG’ 06) • Pathwidth: – – paths: poly-time pathwidth 1: maybe algorithm from EGG’ 06 still works pathwidth 2: any KLS-style algo is exptime (EGG’ 06) pathwidth > r, r constant: PPAD-complete (EGG’ 06) • Finding NE on trees?