ARTIFICIAL INTELLIGENCE Games GAMES AND COMPUTERS Games offer

ARTIFICIAL INTELLIGENCE Games

GAMES AND COMPUTERS Games offer concrete or abstract competitions “I’m better than you!” Some games are amenable to computer treatment mostly mental activities well-formulated rules and operators accessible state Others are not emphasis on physical activities rules and operators open to interpretation state not (easily or fully) accessible 2

DEFINING COMPUTER GAMES Video game / computer game “a mental contest, played with a computer according to certain rules for amusement, recreation, or winning a stake” [Zyda, 2005] Serious game a mental contest, played with a computer in accordance with specific rules, that uses entertainment to further training or education [modified from Zyda, 2005] [Zyda, 2005] Michael Zyda, “From Visual Simulation to Virtual Reality to Games, ” IEEE Computer, vol. 39, no. 9, pp. 25 -32. 3

WHAT ARE AND WHY STUDY GAMES? Games are a form of multi-agent environment What do other agents do and how do they affect our success? Cooperative vs. competitive multi-agent environments. Competitive multi-agent environments give rise to adversarial search a. k. a. games Why study games? Fun Interesting subject of study because they are hard 4

RELATION OF GAMES TO SEARCH Search – no adversary Solution is (heuristic) method for finding goal Heuristics and CSP techniques can find optimal solution Evaluation function: estimate of cost from start to goal through given nodes Examples: path planning, scheduling activities Games – adversary Solution is strategy (strategy specifies move for every possible opponent reply). Time limits force an approximate solution Evaluation function: evaluate “goodness” of game position Examples: chess, checkers, Othello, backgammon 5

ASPECTS OF COMPUTER GAMES Story defines the context and content of the game Art presentation of the game to the user emphasis on visual display, sound Software implementation of the game on a computer Purpose entertainment training education 6

EXAMPLES 7

EXAMPLES 8

GAME ANALYSIS Often deterministic the outcome of actions is known sometimes an element of chance is part of the game Two-player, turn-taking one move for each player Zero-sum utility function e. g. dice what one player wins, the other must lose Often perfect information fully observable, everything is known to both players about the state of the environment (game) not for all games e. g. card games with “private” or “hidden” cards 9

TYPES OF GAMES 10

GAMES AS ADVERSARIAL SEARCH Many games can be formulated as search problems The zero-sum utility function leads to an adversarial situation in order for one agent to win, the other necessarily has to lose Factors complicating the search task potentially huge search spaces elements of chance multi-person games, teams time limits 11

SINGLE-PERSON GAME Conventional search problem identify a sequence of moves that leads to a winning state examples: Solitaire and Rubik’s cube little attention in AI Some games can be quite challenging some versions of solitaire 12

CONTINGENCY PROBLEM Uncertainty due to the moves and motivations of the opponent tries to make the game as difficult as possible for the player attempts to maximize its own, and thus minimize the player’s utility function value 13

TWO-PERSON GAME Games with two opposing players often called MIN and MAX usually MAX moves first, then they take turns MAX no MIN or wants a strategy to find a winning state matter what MIN does the same at least tries to prevent MAX from winning Perfect Decisions traverse all relevant parts of the search tree often impractical because of time and space limitations 14

EXAMPLE TRIVIAL GAME Deal four playing cards out, face up Player 1 chooses one, player 2 chooses one Player 1 chooses another, player 2 chooses another And the winner is…. Add the cards up The player with the highest even number Scores that amount 15

ENTIRE SEARCH SPACE 16

MOVING THE SCORES FROM THE BOTTOM TO THE TOP 17

MOVING A SCORE WHEN THERE’S A CHOICE Use minimax assumption 18

CHOOSING THE BEST MOVE 19

MINIMAX STRATEGY Optimal strategy for MAX generate the whole game tree calculate the value of each terminal state starting from the leaf nodes up to the root MAX selects the value with the highest node MAX assumes that MIN in its move will select the node that minimizes the value assumes that both players play optimally 20

MINIMAX PROPERTIES Based on depth-first recursive implementation Time complexity is O(bm) exponential in the number of moves Space complexity is O(bm) where b is the branching factor, m the maximum depth of the search tree 21

MINIMAX EXAMPLE 4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 terminal nodes: values calculated from the utility function 22 9 3

MINIMAX EXAMPLE 4 7 6 2 6 3 4 5 1 2 5 4 1 2 6 3 4 3 Min 4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 other nodes: values calculated via minimax algorithm 23 9 3

MINIMAX EXAMPLE 7 4 7 6 6 2 6 5 3 4 5 5 1 2 5 6 4 1 4 2 6 Max 3 4 3 Min 4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 24 9 3

MINIMAX EXAMPLE 5 7 4 7 3 6 6 2 6 4 5 3 4 Min 5 5 1 2 5 6 4 1 4 2 6 Max 3 4 3 Min 4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 25 9 3

MINIMAX EXAMPLE 5 5 7 4 7 3 6 6 2 6 Max 4 5 3 4 Min 5 5 1 2 5 6 4 1 4 2 6 Max 3 4 3 Min 4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 26 9 3

MINIMAX EXAMPLE 5 5 7 4 7 3 6 6 2 6 Max 4 5 3 4 Min 5 5 1 2 5 6 4 1 4 2 6 Max 3 4 3 Min 4 7 9 6 9 8 8 5 6 7 5 2 3 2 5 4 moves by Max and countermoves by Min 27 9 3

IMPERFECT DECISIONS Complete search is impractical for most games Alternative: search the tree only to a certain depth requires a cutoff-test to determine where to stop uses a heuristics-based evaluation function to estimate the expected utility of the game from the leave nodes 28

PRUNING Discards parts of the search tree guaranteed not to contain good moves guarantee that the solution is not in that branch or sub-tree if both players make optimal decisions, they will never end up in that part of the search tree Results in substantial time and space savings as a consequence, longer sequences of moves can be explored 29

ALPHA-BETA PRUNING Certain moves are not considered won’t result in a better evaluation value Applies to moves by both players a indicates the best choice for Max so far never decreases b indicates the best choice for Min so far never increases Extension of the minimax approach results in the same move as minimax, but with less overhead prunes uninteresting parts of the search tree 30

ALPHA-BETA EXAMPLE 1 [-∞, +∞] 5 Max [-∞, +∞] a best choice for Max b best choice for Min ? ? we assume a depth-first, left-to-right search as basic strategy the range of the possible values for each node are indicated initially [-∞, +∞] 31

ALPHA-BETA EXAMPLE 2 [-∞, +∞] 5 Max [-∞, 7] Min 7 a best choice for Max b best choice for Min ? 7 Min obtains the first value from a successor node 32

ALPHA-BETA EXAMPLE 3 [-∞, +∞] 5 Max [-∞, 6] 7 Min 6 a best choice for Max b best choice for Min ? 6 Min obtains the second value from a successor node 33

ALPHA-BETA EXAMPLE 4 [5, +∞] 5 7 6 5 Max Min 5 a best choice for Max b best choice for Min 5 5 Min obtains the third value from a successor node this is the last value from this sub-tree, and the exact value is known Max now has a value for its first successor node, but hopes that something better might still come 34

ALPHA-BETA EXAMPLE 5 [5, +∞] 5 7 6 5 Max [-∞, 3] 5 Min 3 a best choice for Max b best choice for Min 5 3 Min continues with the next sub-tree, and gets a better value Max has a better choice from its perspective, however, and will not consider a move in the sub-tree currently explored by Min initially [-∞, +∞] 35

ALPHA-BETA EXAMPLE 6 [5, +∞] 5 7 6 [-∞, 3] 5 5 Max Min 3 a best choice for Max b best choice for Min 5 3 Min knows that Max won’t consider a move to this sub-tree, and abandons it this is a case of pruning, indicated by 36

ALPHA-BETA EXAMPLE 7 [5, +∞] 5 7 6 [-∞, 3] 5 3 a best choice for Max b best choice for Min 5 Max [-∞, 6] Min 6 5 3 Min explores the next sub-tree, and finds a value that is worse than the other nodes at this level if Min is not able to find something lower, then Max will choose this branch, so Min must explore more successor nodes 37

ALPHA-BETA EXAMPLE 8 [5, +∞] 5 7 6 [-∞, 3] 5 3 a best choice for Max b best choice for Min 5 Max [-∞, 5] 6 Min 5 5 3 Min is lucky, and finds a value that is the same as the current worst value at this level Max can choose this branch, or the other branch with the same value 38

ALPHA-BETA EXAMPLE 9 5 5 7 6 [-∞, 3] 5 3 a best choice for Max b best choice for Min Max [-∞, 5] 6 Min 5 5 3 Min could continue searching this sub-tree to see if there is a value that is less than the current worst alternative in order to give Max as few choices as possible 5 this depends on the specific implementation Max knows the best value for its sub-tree 39

ALPHA-BETA EXAMPLE OVERVIEW 55 5 7 6 <= 3 5 3 a best choice for Max b best choice for Min Max 6 <=5 4 6 Min 5 4 5 7 -> 6 -> 5 -> 3 some branches can be pruned because they would never be considered after looking at one branch, Max already knows that they will not be of interest since Min would choose a value that is less than what Max already has at its disposal 40