Learning in Games ChiJen Lu Academia Sinica Outline

  • Slides: 32
Download presentation
Learning in Games Chi-Jen Lu Academia Sinica

Learning in Games Chi-Jen Lu Academia Sinica

Outline What machine learning can do for game theory What game theory can do

Outline What machine learning can do for game theory What game theory can do for machine learning

TWO-PLAYER ZERO-SUM GAMES

TWO-PLAYER ZERO-SUM GAMES

Zero-sumgames player 2 player 1 0 -1 1 0 1 -1 1 0 -1

Zero-sumgames player 2 player 1 0 -1 1 0 1 -1 1 0 -1 -1 0 utility (reward) of player 1 utility (reward) of player 2

Zero-sumgames < in many games

Zero-sumgames < in many games

Zero-sumgames distributions

Zero-sumgames distributions

ONLINE LEARNING

ONLINE LEARNING

Online learning / decision Making decisions/predictions repeatedly and then paying the prices I wish

Online learning / decision Making decisions/predictions repeatedly and then paying the prices I wish I had…

Many examples Predicting weather, trading stocks, commuting to work, … Network routing Scheduling Resource

Many examples Predicting weather, trading stocks, commuting to work, … Network routing Scheduling Resource allocation Online advertising …

Problem formulation distribution over K

Problem formulation distribution over K

Goal: minimize regret Regret: total reward of best fixed strategy total reward of online

Goal: minimize regret Regret: total reward of best fixed strategy total reward of online algorithm I wish I had…

No-regretalgorithms no regret

No-regretalgorithms no regret

Applications in other areas algorithms: approximation algorithms complexity: hardcore set for derandomization optimization: LP

Applications in other areas algorithms: approximation algorithms complexity: hardcore set for derandomization optimization: LP duality biology: evolution game theory: minimax theorem 13

Zero-sumgames distributions huge? oneshot game

Zero-sumgames distributions huge? oneshot game

INFLUENCE MAXIMIZATION GAMES

INFLUENCE MAXIMIZATION GAMES

Opinion formation in social net A population of n individuals, each with some internal

Opinion formation in social net A population of n individuals, each with some internal opinion from [-1, 1] vs. Each tries to express an opinion close to neighbors’ opinions and her internal one 16

Opinion formation in social net Zero-sum game between and player/party: ◦ goal: makes n

Opinion formation in social net Zero-sum game between and player/party: ◦ goal: makes n shades of grey darker lighter ◦ actions: controls the opinions of k individuals Find minimax strategy? 17

Opinion formation in social net Zero-sum game between and player/party: ◦ goal: makes n

Opinion formation in social net Zero-sum game between and player/party: ◦ goal: makes n shades of grey darker lighter ◦ actions: controls the opinions of k individuals Find minimax strategy? Solution: no-regret algorithm for online combinatorial optimization. follow the perturbed leader 18

MARKOV GAMES

MARKOV GAMES

Games with states board configurations policy: states actions (randomized) Minimax theorem: policy

Games with states board configurations policy: states actions (randomized) Minimax theorem: policy

Games with states board configurations policy huge?

Games with states board configurations policy huge?

Games with states Solution: no-regret algorithm for twoplayer Markov decision process Time, space poly(#(states),

Games with states Solution: no-regret algorithm for twoplayer Markov decision process Time, space poly(#(states), #(actions)) still huge for many games

Outline What machine learning can do for game theory What game theory can do

Outline What machine learning can do for game theory What game theory can do for machine learning

ALGORITHMS VS. ADVERSARIES

ALGORITHMS VS. ADVERSARIES

No-regret algorithm log T ? find adversarial c benign class of c smaller regret

No-regret algorithm log T ? find adversarial c benign class of c smaller regret

More generally…

More generally…

GENERATIVE ADVERSARIAL NETWORKS

GENERATIVE ADVERSARIAL NETWORKS

Learning generative models fake images!

Learning generative models fake images!

Learning generative models fake images!

Learning generative models fake images!

Learning generative models Training data: real face images novel / fake Learn generative model

Learning generative models Training data: real face images novel / fake Learn generative model G: random seeds face images

Learning generative models novel / fake

Learning generative models novel / fake

Play the zero-sum game fake Still not an easy task! G, D: deep neural

Play the zero-sum game fake Still not an easy task! G, D: deep neural nets. real huge action sets