CS 534 Spring 2019 Adversarial Search Game Playing

  • Slides: 42
Download presentation
CS 534 Spring 2019 Adversarial Search, Game Playing Showcase by: Varun Bhat, Ruofan Hu,

CS 534 Spring 2019 Adversarial Search, Game Playing Showcase by: Varun Bhat, Ruofan Hu, Jiayi Li, Justin Seeley, and Matthew Szpunar Showcasing work by: David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel & Demis Hassabis on Alpha. Go

References and Resources 2 CS 534 Artificial Intelligence: Bhat - Hu - Li -

References and Resources 2 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Outline 4 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley -

Outline 4 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Alpha. Go Introduction

Alpha. Go Introduction

What is Alpha. Go? Game of Go in Progress https: //www. mastersofgames. com/images/orientalboard/go-tableboard-pay. jpg

What is Alpha. Go? Game of Go in Progress https: //www. mastersofgames. com/images/orientalboard/go-tableboard-pay. jpg 6 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

What is Alpha. Go? Position for Black to Capture https: //online-go. com/puzzle/14036 7 CS

What is Alpha. Go? Position for Black to Capture https: //online-go. com/puzzle/14036 7 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

What is Alpha. Go? Position for Black to Capture https: //online-go. com/puzzle/14036 8 CS

What is Alpha. Go? Position for Black to Capture https: //online-go. com/puzzle/14036 8 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

What is Alpha. Go? Screenshot of Alpha. Go playing Lee Sidol (2016) https: //s

What is Alpha. Go? Screenshot of Alpha. Go playing Lee Sidol (2016) https: //s 3 -ap-south-1. amazonaws. com/av-blog-media/wpcontent/uploads/2017/01/09112900/alphago-vs-lee-sedol-2_w_600. jpg 9 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Alpha. Go vs. other Go AI • The strong Go AIs all rely on

Alpha. Go vs. other Go AI • The strong Go AIs all rely on Monte Carlo Tree Search (MCTS). • Alpha. Go however makes extensive use of machine learning to avoid using hand-crafted rules. Various Go AI vs. Skill Ranking (ELO) 10 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar “Mastering the Game of Go with Deep Neural Networks and Tree Search. ” (2017)

Alpha. Go Algorithms

Alpha. Go Algorithms

Monte Carlo Tree Search(MCTS) 12 CS 534 Artificial Intelligence: Bhat - Hu - Li

Monte Carlo Tree Search(MCTS) 12 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS.

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 13 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS.

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 14 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS.

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 15 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS.

How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 16 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How to make MCTS work for GO? picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 17 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? 18 CS 534 Artificial Intelligence: Bhat -

How to make MCTS work for GO? 18 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? Main idea for Alpha. Go: For better

How to make MCTS work for GO? Main idea for Alpha. Go: For better policies and value functions, use Convolutional Neural Networks to train. 19 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Large search tree 20 CS 534 Artificial Intelligence: Bhat - Hu - Li -

Large search tree 20 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Convolution Neural Network 21 CS 534 Artificial Intelligence: Bhat - Hu - Li -

Convolution Neural Network 21 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Policy-network (Supervised learning) 22 CS 534 Artificial Intelligence: Bhat - Hu - Li -

Policy-network (Supervised learning) 22 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Value network 23 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley

Value network 23 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Reinforcement learning picture taken from medium. com 24 CS 534 Artificial Intelligence: Bhat -

Reinforcement learning picture taken from medium. com 24 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Reinforcement learning - training picture taken from medium. com 25 CS 534 Artificial Intelligence:

Reinforcement learning - training picture taken from medium. com 25 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Thank you! Questions

Thank you! Questions

Backup Slides

Backup Slides

How does Reinforcement Learning Work picture taken from Geeksfor. Geeks 28 CS 534 Artificial

How does Reinforcement Learning Work picture taken from Geeksfor. Geeks 28 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Go Positions according to policies 29 CS 534 Artificial Intelligence: Bhat - Hu -

Go Positions according to policies 29 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How to make MCTS work for GO? picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How to make MCTS work for GO? picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 30 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 31 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 32 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work 33 CS 534 Artificial Intelligence: Bhat -

How does Monte Carlo Tree Search Work 33 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 34 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 35 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 36 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 37 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work 38 CS 534 Artificial Intelligence: Bhat -

How does Monte Carlo Tree Search Work 38 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 39 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917

How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 40 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Increased Proficiency with More Power Elo Ranking 41 CS 534 Artificial Intelligence: Bhat -

Increased Proficiency with More Power Elo Ranking 41 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

Convolutional Neural Networks 42 CS 534 Artificial Intelligence: Bhat - Hu - Li -

Convolutional Neural Networks 42 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar

CS 534 Spring 2019 Adversarial Search Game PlayingCS 534 Spring 2019 Adversarial Search Game Playing
Adversarial Search Adversarial Search Game playing Perfect playAdversarial Search Adversarial Search Game playing Perfect play
Adversarial Search and Game Playing Examples Game TreeAdversarial Search and Game Playing Examples Game Tree
Adversarial Search game playing search We have experienceAdversarial Search game playing search We have experience
Game Playing Chapter 5 Game playing Search appliedGame Playing Chapter 5 Game playing Search applied
Adversaries Adversarial examples Adversarial examples Ostrich Adversarial examplesAdversaries Adversarial examples Adversarial examples Ostrich Adversarial examples
Game Playing Adversarial Search Chapter 6 Why studyGame Playing Adversarial Search Chapter 6 Why study
Adversarial Search Game Playing Chapter 6 Outline GamesAdversarial Search Game Playing Chapter 6 Outline Games
Game Playing Adversarial Search Chapter 6 Why studyGame Playing Adversarial Search Chapter 6 Why study
Adversarial Search and Game Playing Where making goodAdversarial Search and Game Playing Where making good
ADVERSARIAL SEARCH GAME PLAYING 2 TEXAS HOLD EMADVERSARIAL SEARCH GAME PLAYING 2 TEXAS HOLD EM
Adversarial Search and Game Playing Russell and NorvigAdversarial Search and Game Playing Russell and Norvig
Game Playing Adversarial Search University of Berkeley USAGame Playing Adversarial Search University of Berkeley USA
Adversarial Search and Game Playing Russell and NorvigAdversarial Search and Game Playing Russell and Norvig
Adversarial Search and Game Playing CHAPTER 5 CMPTAdversarial Search and Game Playing CHAPTER 5 CMPT
Game Playing Adversarial Search University of Berkeley USAGame Playing Adversarial Search University of Berkeley USA
Chapter 5 Adversarial Search Game Playing Typical assumptionsChapter 5 Adversarial Search Game Playing Typical assumptions
Adversarial Search and Game Playing Where making goodAdversarial Search and Game Playing Where making good
Game Playing Adversarial Search Chapter 5 Why studyGame Playing Adversarial Search Chapter 5 Why study
Lecture 02 Part C Game Playing Adversarial SearchLecture 02 Part C Game Playing Adversarial Search
Game Playing Chapter 6 Game Playing and AIGame Playing Chapter 6 Game Playing and AI
Pengantar Kecerdasan Buatan Game Playing 2 Game PlayingPengantar Kecerdasan Buatan Game Playing 2 Game Playing
Game Playing Chapter 6 Game Playing and AIGame Playing Chapter 6 Game Playing and AI
Ni dung chnh Game playing Game playing problemNi dung chnh Game playing Game playing problem