CS 534 Spring 2019 Adversarial Search Game Playing
- Slides: 42
CS 534 Spring 2019 Adversarial Search, Game Playing Showcase by: Varun Bhat, Ruofan Hu, Jiayi Li, Justin Seeley, and Matthew Szpunar Showcasing work by: David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel & Demis Hassabis on Alpha. Go
References and Resources 2 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Outline 4 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Alpha. Go Introduction
What is Alpha. Go? Game of Go in Progress https: //www. mastersofgames. com/images/orientalboard/go-tableboard-pay. jpg 6 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
What is Alpha. Go? Position for Black to Capture https: //online-go. com/puzzle/14036 7 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
What is Alpha. Go? Position for Black to Capture https: //online-go. com/puzzle/14036 8 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
What is Alpha. Go? Screenshot of Alpha. Go playing Lee Sidol (2016) https: //s 3 -ap-south-1. amazonaws. com/av-blog-media/wpcontent/uploads/2017/01/09112900/alphago-vs-lee-sedol-2_w_600. jpg 9 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Alpha. Go vs. other Go AI • The strong Go AIs all rely on Monte Carlo Tree Search (MCTS). • Alpha. Go however makes extensive use of machine learning to avoid using hand-crafted rules. Various Go AI vs. Skill Ranking (ELO) 10 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar “Mastering the Game of Go with Deep Neural Networks and Tree Search. ” (2017)
Alpha. Go Algorithms
Monte Carlo Tree Search(MCTS) 12 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 13 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 14 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 15 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? picture from http: //www. yisongyue. com/courses/cs 159/lectures/MCTS. pdf 16 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 17 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? 18 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? Main idea for Alpha. Go: For better policies and value functions, use Convolutional Neural Networks to train. 19 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Large search tree 20 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Convolution Neural Network 21 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Policy-network (Supervised learning) 22 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Value network 23 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Reinforcement learning picture taken from medium. com 24 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Reinforcement learning - training picture taken from medium. com 25 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Thank you! Questions
Backup Slides
How does Reinforcement Learning Work picture taken from Geeksfor. Geeks 28 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Go Positions according to policies 29 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How to make MCTS work for GO? picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 30 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 31 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 32 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work 33 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 34 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 35 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 36 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 37 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work 38 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 39 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
How does Monte Carlo Tree Search Work picture from https: //towardsdatascience. com/monte-carlo-tree-search-158 a 917 a 8 baa 40 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Increased Proficiency with More Power Elo Ranking 41 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar
Convolutional Neural Networks 42 CS 534 Artificial Intelligence: Bhat - Hu - Li - Seeley - Szpunar