Pengantar Kecerdasan Buatan Game Playing 2 Game Playing

Pengantar Kecerdasan Buatan Game Playing

2 Game Playing & AI • Game playing (was? ) thought to be a good problem for AI research: players need “human-like” intelligence games can be very complex (e. g. , chess, go) requires decision making within limited time games usually are: • well-defined and repeatable • limited and accessible • can directly compare humans and computers • • • Game AI is also known as adversarial search problem OSCAR KARNALIM, S. T. , M. T.

3 Games vs Search Problems • Unpredictable opponent • • Time limits • • Solution is a strategy specifying a move for every possible opponent reply Unlikely to find goal, must approximate Plan of attack: • • • Computer considers possible lines of play (Babbage, 1846) Algorithm for perfect play (Zermelo, 1912; Von Neumann, 1944) Finite horizon, approximate evaluation (Zuse, 1945; wiener, 1948; Shannon, 1950) First chess program (Turing, 1951) Machine learning to improve evaluation accuracy (Samuel, 1952 -57) Pruning to allow deeper search (Mc. Carthy, 1956) OSCAR KARNALIM, S. T. , M. T.

4 Types of Game • Perfect vs Imperfect information: Perfect: See the exact state of the games • Imperfect: Information is hidden • • Deterministic vs Change: Deterministic: Change in state is fully determined by player move. • Change: Change in state is partially determined by chance. • OSCAR KARNALIM, S. T. , M. T.

5 Types of Game (Cont) Perfect Information Imperfect Information Deterministic chess, checkers, go, othello battleships, blind tictactoe Change backgammon monopoly bridge, poker, scrabble nuclear war OSCAR KARNALIM, S. T. , M. T.

6 Game Playing as Search • Consider a two player board game: e. g. , chess, checkers, tic-tac-toe • board configuration: unique arrangement of "pieces“ • • Representing board games as search problem: • • • states: board configurations operators: legal moves initial state: current board configuration terminal state: winning/terminal board configuration utility function: values for terminal state (win: +1, loss: -1, draw: 0) We want to find a strategy (i. e. way of picking moves) that wins the game. OSCAR KARNALIM, S. T. , M. T.

7 Game Tree (2 -player, deterministic, turns) OSCAR KARNALIM, S. T. , M. T.

8 Game Playing as Complexity Assume the opponent’s moves can be predicted given the computer's moves How complex would search be in this case? worst case: O(bm) branching factor, max depth Chess: ~35 legal moves, ~100 moves per game Common games produce enormous search trees OSCAR KARNALIM, S. T. , M. T.

9 Minimax • Expand complete search tree in DFS manner, until terminal states have been reached and their utilities computed. • Computer favors high utility value and the opponent favors low utility value. Computer will choose the moves that maximize the utility value. • Based on zero-sum concept OSCAR KARNALIM, S. T. , M. T.

10 Minimax Example : Tic Tac Toe • Reference : http: //neverstopbuilding. com/minimax • End game conditions: I win, hurray! I get 10 points! • I lose, shit. I lose 10 points (because the other player gets 10 points) • I draw, whatever. I get zero points, nobody gets any points. • • X : computer’s symbol in game • O : opponment’s symbol in game OSCAR KARNALIM, S. T. , M. T.

11 Minimax Example : Tic Tac Toe (Cont) • Pada giliran X, tujuan X adalah memaksimalkan utility function • X diharapkan akan memilih langkah dia menang • Skor terbesar OSCAR KARNALIM, S. T. , M. T.

12 Minimax Example : Tic Tac Toe (Cont) • Pada giliran O, tujuan O adalah meminimalkan utility function • O akan diharapkan memilih langkah dia kalah • Skor terkecil OSCAR KARNALIM, S. T. , M. T.

13 Minimax Example : Tic Tac Toe (Cont) • If the game is over, return the score from X's perspective. • Otherwise • • • get a list of new game states for every possible move Create a scores list For each of these states add the minimax result of that state to the scores list If it's X's turn, return the maximum score from the scores list If it's O's turn, return the minimum score from the scores list OSCAR KARNALIM, S. T. , M. T.

14 Minimax Example : Tic Tac Toe (Cont) OSCAR KARNALIM, S. T. , M. T.

15 Minimax Example : Tic Tac Toe (Cont) • It's X's turn in state 1. X generates the states 2, 3, and 4 and calls minimax on those states. • State 2 pushes the score of +10 to state 1's score list • • because the game is in an end state. State 3 and 4 are not in end states 3 generates states 5 and 6 and calls minimax on them • 4 generates states 7 and 8 and calls minimax on them. • OSCAR KARNALIM, S. T. , M. T.

16 Minimax Example : Tic Tac Toe (Cont) • State 5 pushes a score of -10 onto state 3's score list • State 7 pushes a score of -10 onto state 4's score list. • State 6 and 8 generate the only available moves, which are end states • both of them add the score of +10 to the move lists of states 3 and 4. OSCAR KARNALIM, S. T. , M. T.

17 Minimax Example : Tic Tac Toe (Cont) • Because it is O's turn in both state 3 and 4, O will seek to find the minimum score, and given the choice between -10 and +10, both states 3 and 4 will yield -10. • Finally the score list for states 2, 3, and 4 are populated with +10, -10 and -10 respectively, and state 1 seeking to maximize the score will chose the winning move with score +10, state 2. OSCAR KARNALIM, S. T. , M. T.

18 Minimax Algorithm OSCAR KARNALIM, S. T. , M. T.

19 Properties of Minimax • Complete? Yes (if tree is finite) • Optimal? Yes (against an optimal opponent) • Time complexity? O(bm) • Space complexity? O(bm) (depth-first exploration) • For chess, b ≈ 35, m ≈100 for "reasonable" games exact solution completely infeasible • But do we need to explore every path? OSCAR KARNALIM, S. T. , M. T.

20 - Pruning idea Some of the branches of the game tree won't be taken if playing against an intelligent opponent • Pruning can be used to ignore those branches • Keep track of these values while doing DFS of game tree: • maximizing level: highest value seen so far • minimizing level: lowest value seen so far • OSCAR KARNALIM, S. T. , M. T.

21 α-β Pruning Example OSCAR KARNALIM, S. T. , M. T.

22 α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

23 α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

24 α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

25 α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

26 Properties of α-β • Pruning does not affect final result • Good move ordering improves effectiveness of pruning OSCAR KARNALIM, S. T. , M. T.

27 Why is it called α-β ? • α is the value of the best (i. e. , highest-value) choice found so far at any choice point along the path for max • If v is worse than α, max will avoid it prune that branch • Define β similarly for min OSCAR KARNALIM, S. T. , M. T.

The α-β Algorithm 28 OSCAR KARNALIM, S. T. , M. T.

29 More Detailed α-β Pruning Example OSCAR KARNALIM, S. T. , M. T.

30 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

31 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

32 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

33 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

34 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

35 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

36 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

37 More Detailed α-β Pruning Example (Cont) α≥β prune OSCAR KARNALIM, S. T. , M. T.

38 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

39 More Detailed α-β Pruning Example (Cont) old β ≥ new β change OSCAR KARNALIM, S. T. , M. T.

40 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

41 More Detailed α-β Pruning Example (Cont) old α ≤ new α change OSCAR KARNALIM, S. T. , M. T.

42 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

43 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

44 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

45 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

46 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

47 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

48 More Detailed α-β Pruning Example (Cont) α≥β prune OSCAR KARNALIM, S. T. , M. T.

49 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

50 More Detailed α-β Pruning Example (Cont) old α ≤ new α change OSCAR KARNALIM, S. T. , M. T.

51 More Detailed α-β Pruning Example (Cont) OSCAR KARNALIM, S. T. , M. T.

52 More Detailed α-β Pruning Example (Cont) α≥β prune OSCAR KARNALIM, S. T. , M. T.

53 Resource Limits • Standard approach: Use Cutoff-Test instead of Terminal-Test e. g. , depth limit (perhaps add quiescence search) • Use Eval instead of Utility i. e. , evaluation function that estimates desirability of position • • Suppose a PC can explore 106 nodes/second, allowing it to search roughly 200 per move (standard time 3 minutes/move) • • Chess branching factor is 355 50 million α-β reaches 10 ply pretty good chess program OSCAR KARNALIM, S. T. , M. T.

54 Nondeterministic Games in General • In nondeterministic games, chance introduced by dice, cardshuffling • Simplified example with coin-flipping: OSCAR KARNALIM, S. T. , M. T.

55 Nondeterministic Games • The Game Tree Representation is Extended: OSCAR KARNALIM, S. T. , M. T.

56 Nondeterministic Games (Cont) • Weight score by the probabilities that move occurs • Use expected value for move (sum of possible random outcomes) OSCAR KARNALIM, S. T. , M. T.

57 Nondeterministic Games (Cont) • Choose move with highest expected value OSCAR KARNALIM, S. T. , M. T.

58 Algorithm for Nondeterministic Games • Expectiminimax gives perfect play • Just like Minimax, except we must also handle chance nodes: OSCAR KARNALIM, S. T. , M. T.

59 Games of Imperfect Information • E. g. , card games, where opponent's initial cards are unknown • Typically we can calculate a probability for each possible deal • Seems just like having one big dice roll at the beginning of the game • Idea: compute the minimax value of each action in each deal, then choose the action with highest expected value over all deals OSCAR KARNALIM, S. T. , M. T.

60 Human or Computers – Who is Better ? • Checkers: 1994: Chinook (U. of A. ) beat world champion Marion Tinsley, ending 40 -yr reign. • • Othello: • 1997: Logistello (NEC research) beat the human world champion. • Today: world champions refuse to play AI computer program (because it’s too good). Chess: • 1997: Deep Blue (IBM) beat world champion Gary Kasparov • 2005: a team of computers (Hydra, Deep Junior and Fritz), wins 8. 5 -3. 5 against a rather strong human team formed by Veselin Topalov, Ruslan Ponomariov and Sergey Karjakin, who had an average ELO rating of 2681. • 2006: The undisputed world champion, Vladimir Kramnik, is defeated 4 -2 by Deep Fritz. Backgammon: • • Go: • • TD-Gammon (IBM) is world champion amongst humans and computers Human champions refuse to play top AI player (because it’s too weak) Bridge: Still out of reach for AI players. OSCAR KARNALIM, S. T. , M. T.

61 Deep Blue • Specialized parallel chess processor (32 nodes, each node has 8 VLSI “chess” chips) special-purpose memory architecture. • Very sophisticated evaluation function (expert features, tuned weights). • Database of standard openings/closings. • Uses a version of α-β pruning • • Can search up to 40 -deep in some branches. Can search over 200 million positions per second! OSCAR KARNALIM, S. T. , M. T.

62 Deep Blue (Cont) Kasparov vs. Deep Blue, May 1997 6 game full-regulation chess match sponsored by ACM • Kasparov lost the match 2 wins & 1 tie to 3 wins & 1 tie • This was an historic achievement for computer chess being the first time a computer became the best chess player on the planet • Note that Deep Blue plays by “brute force” (i. e. , raw power from computer speed and memory); it uses relatively little that is similar to human intuition and cleverness • OSCAR KARNALIM, S. T. , M. T.

63 Chinook (Checkers) • Plain α-β search, performed on standard PCs. • Evaluation function based on expert features of the board. • Opening database, HUGE end game database. • • Including full search tree for all checkers positions involving or fewer pieces on the board (443, 748, 401, 247 positions!!!) Only a few moves in the middle of the game are actually searched. OSCAR KARNALIM, S. T. , M. T.

64 Logistello (Othello) Database of openings, continuously updated (~23, 000 games so far) • α-β search with a linear evaluation function • • Hand-selected features and weights tuned by learning Thinks during the opponent’s time. Search speed (on Pentium-Pro 200) ≈ 160, 000 nodes/sec in the middle game ≈ 480, 000 nodes/sec in the end game • Search speed ≈ 18 -23 deep in the middle game OSCAR KARNALIM, S. T. , M. T.

65 Go (Chinese: 圍棋 wéiqí, Japanese: 囲碁 igo) Why Go is hard ? Branching factor 19 x 19 = 351! The placement of a single stone in the initial phase can affect the play of the game a hundred or more moves later. This means search is very difficult. The best Go programs manage to reach amateur dan level. On the small 9× 9 board, the computer fares better, and some programs now win a fraction of their 9× 9 games against professional players. require more elements that mimic human thought than chess. Challenge: $2 million prize for any system that can beat a world expert OSCAR KARNALIM, S. T. , M. T.

66 Kesimpulan • Games are fun to work on! (and dangerous) • They illustrate several important points about AI • perfection is unattainable • must approximate good idea to think about what to think about • uncertainty constrains the assignment of values to states • optimal decisions depend on information state, not real state • OSCAR KARNALIM, S. T. , M. T.

67 Referensi • Russell, Stuart J. & Norvig, Peter. Artificial Intelligence: A Modern Approach (3 rd ed. ). Prentice Hall. 2009 • Luger, George F. Artificial Intelligence: Structures and Strategies for Complex Problem Solving. 6 th Edition. Addison Wesley. 2008. • Watson, Mark. , Practical Artificial Intelligence Programming in Java, Open Content – Free e. Book (CC License), 2005. OSCAR KARNALIM, S. T. , M. T.

68 Coba Pohon Permainan Berikut OSCAR KARNALIM, S. T. , M. T.

69 ? ? ? http: //homepage. ufp. pt/jtorres/ensino/ia/alfabeta. html OSCAR KARNALIM, S. T. , M. T.