Performance analysis of AlphaBeta Pruning n n n

Performance analysis of Alpha-Beta Pruning n n n Since alpha-beta pruning performs a minimax search while pruning much of the tree, its effect is to allow a deeper search with the same amount of computation. The question: how much does alpha-beta improve performance? The best way to characterize is asymptotic effective branching factor. The dth root of the number of nodes (in a search to depth d, in the limit of large d) u number of nodes generated at depth d / number of nodes generated at depth d-1. u

Performance analysis of Alpha-Beta Pruning n n The efficiency of alpha-beta pruning depends upon the order in which nodes are encountered at the search frontier. Thus, we consider 3 different cases: u worst case - the algorithm doesn’t perform any cutoffs at all u best case u average case

n Example of alpha-beta worst case Evaluation from left to right MAX 4 MIN 4 2 4 4 4 5 8 2 6 2 8 1 14 2 12 14 3 2 6 7 8 9 1 10 2 11 12 13 14 14

Lower Bound for Minimax Algorithms n n n We consider a lower bound on the number of leaf nodes that must be examined by any minimax algorithm. In minimax algorithm, it’s a guaranty to return the minimax value v of the root node of a game tree. verifying maximum value = v verifying value v && value v. +Any correct minimax algorithm must explore: ua strategy for Max u a strategy for Min

Strategies for Min and Max value v: n doesn’t matter what min does n Strategy for max: max subtree containing: t one child of each Max node t all b children of each min node value v: n doesn’t matter what Max does n Strategy for min: min subtree containing: t one child of each Min node t all b children of each Max node

Example strategy for Min: strategy for Max: mixed Max strategy Min strategy

Lower Bound for Minimax Algorithms - Analysis Assume : n uniform branching factor of b uniform depth of d levels n Max move is at the root. n Strategy for Max t t d is even nodes d is odd nodes Strategy for Min leaf t d is even nodes d is odd nodes leaf

Lower Bound for Minimax Algorithms - Analysis Total number of distinct leaf nodes: nodes n n d is odd : d is even : note: n there is a single leaf node in common of both strategies.

Lower Bound for Minimax Algorithms - Analysis b n n d/2 + b d/2 -1 = O(bd/2 ) This is the number of leaf nodes that must be examined by any minimax algorithm. This is the lower bound of the time complexity.

Minimax value of game trees n n n The most natural definition for the average case is that the leaf nodes are randomly ordered. Heuristic node ordering would violate this assumption. Average case performance is not a prediction of its performance in practice

Minimax value of game trees n The root will be in the average case of randomly ordered frontier nodes. Special case u leaf nodes: F are actual terminal positions, F have the exact values of WIN or LOSS. n Most general case u arbitrary leaf node values

WIN-LOSS Trees analytic model n n n uniform branching factor b uniform depth d Max is to move at the root depth d is even terminal nodes labeled WIN with probability P 0 terminal nodes labeled LOSS with probability 1 - P 0

Example: Board Splitting n n n Two players: vertical and horizontal square sheet of graph paper, bd/2 squares on each side each square V with probability P 0 and H with probability 1 - P 0 n vertical’s turn: turn divides the board vertically into b equals slices, discarding all but one of them.

Board Splitting n n horizontal’s turn: divides the board turn horizontally into b equals slices, discarding all but one of them. Result: Result the initial in the only square left indicates the winner.

Complexity of WIN-LOSS Tree n Pn Pn probability that Max force a win, given that Max is to move Qn 2 n moves in the tree n Qn probability that Max force a win, given that min is to move 2 n -1 moves in the tree Pn-1 Max min

WIN-LOSS Trees This is the probability that a Max node at any higher level in the tree will be a win for Max

WIN-LOSS Trees Min is to move to be a win for Max. all of its children must be wins for Max t probability that all b children of a node are win for Max Q n = (Pn - 1)b t Max is to move to be a loss for Max all of its children must be losses for Max t probability that a node is a loss for Max 1 - probability that it is a win for Max 1 - P n = (1 - Q n) t

WIN-LOSS Trees 1. 00 crossover point: determines the probability of a win for Max 0. 90 0. 80 0. 70 0. 60 0. 50 0. 40 0. 30 0. 20 0. 10 0. 00 ב 0. 00 0. 20 0. 40 0. 60 0. 80 Graph showing iterations of function f(x) = 1 - (1 - x 2)2. 1. 00

WIN-LOSS Trees If the probability of a win for MAX at the leaves is grater than crossover point, then the large enough game tree is almost certainly a forced win for MAX!

WIN-LOSS Trees n n Let be is the fixed point of the iteration (crossover point). For b = 2, This value is also known as “golden ratio”. The probability that the root of a game tree is a forced win for Max: Even through wins and losses are chosen randomly at the leaf nodes, we can predict the win-loss status of the root of a sufficiently deep minimax tree with almost certainty, simply by knowing the probability of a win at the leaves!

Minimax convergence theorem n We now generalize our result to the case of leaf nodes with arbitrary numerical values. We adopt a following model: u Uniform branching factor b u uniform depth d u leaves are assigned random numeric value, but from a common probability distribution function Fv 0(v) = P(v 0 v). u v 0 is a particular node value chosen from this distribution

Minimax convergence theorem n Let’s determine the probability distribution of the minimax value of the root of a tree u in the limit of large depth u as a function of the probability distribution of the leaf values u with the expression of the distribution of the minimax values at 2 n levels above the leaves as Fvn(v)

Minimax convergence theorem For any value of v: n a leaf node is a win for Max a leaf node is a win for Min its value > v its value >v Minimax value of a max node will be greater than v any one of its children has a minimax value greater than v. F The n Minimax value of a min node will be greater than v all of its children has a minimax value greater than v. minimax values propagate up the same as in the winloss trees.

Minimax convergence theorem n n In win-loss tree Pn is the probability that a Max node 2 n levels above the leaves is a forced win for Max. In general game tree The theorem:

Minimax convergence theorem The Meaning: n Probability distribution is zero up to a particular value of v (v*). in a b-ary tree: u n beyond v*, the probability distribution function of the minimax value of the root is 1. The probability density function of the step distribution function Fvn(v) is an impulse at v*. All the probability mass is concentrated at v *.

Conclusion for a minimax tree We can predict exactly what the minimax value of the root of the tree will be. U Given: F arbitrary terminal values chosen independently from the same distribution F limit of large depth

Example (application) Goal: Goal n a fuse that will burn out after a specific time. problem: n we only have fuses that have same broad distribution of burn-out time. solution: n we connect two fuses in parallel the burn-out time of the whole circuit will be the maximum of the burn-out time of the individual fuses u the circuit will remain closed until both fused burn-out u n The burn-out time of the entire circuit is the minimax value of the burn-out times of the individuals fuses.

Average-case time complexity Win-Loss game tree We assume the previous model. Assume that According minimax the to convergence theorem, at sufficiently high levels of the tree, all nodes are losses for Max and wins for Min. n Max node all the children must be examined all the children will be a loss for Max Min node only the first node must be examined it will be a win for Max

Average-case time complexity Win-Loss game tree If we follow any path from root, we will branch: u only one way at the alternating Max levels u b ways at every other level The asymptotic number of leaf nodes in the limit of large depth O(bd/2 ) @ effective branching factor of b 1/2

Average-case time complexity Win-Loss game tree Assume sufficiently case, atthisthat In high levels of the tree, all nodes are losses for Max and wins for Min. u As the above case, this alse results in effective b. f of b 1/2 ¸ Assume that u Pearl (extremely rare case) shows : effective branching factor =

Average-case time complexity - Trees with arbitrary terminal values n There are two possibilities for choosing the leaf values: ua continuous distribution - segment of the real number line. F Minimax values of all nodes will be equal F Alpha-beta pruning will realize its best performance ua discrete distribution - only a finite number of distinct values. F The probability that any node takes an any particular value is zero F Pearl shows:

Introduction n We generalize the 2 -player-perfect-information algorithms, to the case of non-cooperative perfectinformation -more players games. n No coalitions between players. n Examples: Examples u Chinese Checkers with 6 players. u Othello extended by having different colored pieces for each player.

Maxn (maxn) Algorithm Assumption: Assumption n the players alternate moves n each player tries to maximize his/her return n and indifferent to returns of others.

Maxn Algorithm n At frontier nodes, evaluation function returns an n -tuple of values: (player 1, P 2, P 3, …. Player n) n For example: u Othello - return number of pieces for each player.

Maxn Algorithm evaluation function in each interior node where player i is to move = the entire n-tuple of the child for which the ith component is maximum.

Maxn Algorithm - Example 1 (7, 3, 6) (1, 7, 2) (7, 3, 6) 2 2 (6, 5, 4) (1, 7, 2) (7, 3, 6) 3 3 (3, 1, 8) (2, 8, 1) (1, 7, 2) (5, 6, 3) (6, 5, 4) (8, 5, 4) (7, 3, 6) (4, 2, 7) (3, 1, 8)

Maxn Algorithm n Formal notations: u u u n M(x) static heuristic value of node x M(x, p) - backed-up maxn value of node x by player p. Mi(x, p) - component of M(x, p) corresponds to the return for player i. M(xi, p’) = max. Mp(xi, p’) over children of node x, p’ is player that follows player p tie breaking in favor of leftmost node. Recursive definition of the maxn node:

Maxn Algorithm n Minimax can be viewed as a special case of maxn, when un = 2, u evaluation function: (x, -x). n Luckhardt & Irani observed: u at nodes where player i is to move, only the ith component of the children need be evaluated. u It may be no less expensive to compute all components. n Without assumptions on values of components, pruning of branches is impossible (with more than 2 players).

Alpha-Beta in multi-Player Games n Tree pruning is possible when : u there is an upper bound on the sum of all components of a tuple u a lower bound on the values of each component. n For example: u Othello - no player can have less than zero, and total number of pieces on the board is equal for all nodes at same level.

Immediate Pruning n n n Player I is to move, and in one child the ith component equals the upper bound of sum on all components. Obvious that any other child can be pruned. This is equivalent to situations in the two-player case when a child of a Max node has value of , or a child of a Min node has value of - , indication a won position for the corresponding player.

Shallow Pruning ) 3, 6( (3, 3, 3) 1 (3, 3, 3) 2 3 (3, 3, 3) 3 (4, 2, 3) ) 2, 7, 2( 3 (3, 1, 5) 2 3 (1, 7, 1) ) 3, 6, 3( 2 3 (1, 6, 2)

Shallow algorithm Shallow(Node, Player, Bound) IF Node is terminal, RETURN static value Best = Shallow(first Child, next Player, Sum) FOR each remaining Child IF Best[Player] >= Bound, RETURN Best Current = Shallow(Child, next Player, Sum - Best[Player] ) IF Current[Player] > Best[Player], Best = Current RETURN Best

Failure of deep pruning n In a 2 -player game, alpha-beta allows deep pruning - pruning a node based on bounds inherited from its great-grandparents, or other distant ancestor. Deep pruning does not generalize to more than 2 players!

Failure of Deep Pruning Example 1 ) 5, 4( 2 2 (5, 2, 2) ) 4, 5( 1 (2, 2, 5) 3 3 1 (2, 3, 4)or (3, 0, 6( (6, 1, 2)

Optimality of shallow pruning n Theorem: u Every directional algorithm that computes the maxn value of a game tree with more than 2 players must evaluate every terminal node evaluated by shallow pruning under the same order. n Steps of proof: u The formal proof is by induction on the height of the tree and generalizes the result to an arbitrary number of players greater than 2.

Optimality of shallow pruning n Try to see a “zipper” effect in the sense that the original order of the “teeth” (nodes) at the bottom determines the order of the teeth at the top, even though no individual tooth can move very far.

Minimax and Pathology n n n So far, we have considered the time and spacre complexities of minimax search. We now turn our attention to the quality of the decisions it makes. Since alpha-beta makes exactly the same decisions as minimax search, the question is the decision quality of minimax.

Exact Terminal Values n n n If the leaf nodes of the tree are evaluated exactly, the minimax makes optimal moves against an opponent who plays perfectly. But decision quality is not optimal against an imperfect opponent, who can make mistakes. Example of situation: u two moves are available: - may lead easily and immediately to loss - require a long sequence of moves and great deal of skill of the opponent to force the loss. u Minimax has no preference for one option over the other.

Exact Terminal Values Example n n Against an infallible opponent, it does no matter what move is made. Against an opponent who can make mistakes, it is far preferable to choose the move that requires the most skill on the part of opponent, increasing the chances of an error by the opponent - and hence a win by the player to move.

Minimaxing of Heuristic values n n With the exception of the endgame, the values associated with most nodes in a game are heuristic values returned by the static evaluation. To decrease an error, the heuristic values should be maximized up as if they were exact values.

Minimaxing of Heuristic values-Example n Consider a Max node with two children: x, y - the static heuristic values of the child nodes m m = max(x, y) (minimax value of the parent node ) x n y Assume that : the true value of each child node is a random variable uniformly distributed from 0 to 1. u the variables are independent. u n The most natural way to estimate their values would be their expected values, which are 1/2.

Minimaxing of Heuristic values-Example n Since the minimax value of the parent node is dependent on the values of its children, m becomes a random variable as well. m = max(x, y) = 1/2 n n A better estimate of the value of the parent would be its expected value, or expected value of the maximum of x and y. Note: The expected value of the maximum of two values is not the same as the maximum of of their expected values!

Minimaxing of Heuristic values-Proof Since x, y~U(0, 1) PDF - probability function pdf - density function

Minimaxing of Heuristic values n n n Thus, the expected value of the maximum of two random variables chosen independently from the uniform distribution from 0 to 1 is 2/3, while the maximum of their expected values is only 1/2. The essential error of minimax is is to take the maximum/minimum of the expected values instead of computing the expected value of the maximum/minimum. As the search deeper, the minimax value accumulate more and more error.

Minimaxing of Heuristic values n Why not to compute the exact value of the root of a game tree? u This requires the exact distribution function of all of our leaf nodes, which we don’t know in a real game. u To do the above calculation, we assumed that the values of the child nodes were independent of one another, which is unlikely to be true in a real game. u Even if we had the exact distribution and they were independent, the distributions of interior nodes become increasingly complex functions and can be calculated exactly only in small trees.

Game tree pathology n n n The above error in maximizing gives rise to an effect known as game-tree pathology In the board-splitting game tree, the decision quality of minimax as a function of search depth increases with increasing depth, up to a point, Beyond a certain depth, the percentage of optimal moves made by minimax searching deeper it less than for minimax searching to a shallower depth. The meaning: The error propagation due to maximizing overcomes the additional information derived from searching deeper.

Game tree pathology n n n In real games, searching deeper almost never results in poorer overall quality of play. The puzzle then is to determine what is about board splitting that causes it to be pathological, unlike real games. There are several possible explanations.

Game tree pathology Possible Reasons

Game tree pathology n We can remove any of the assumptions of board splitting, as: uniform branching factor u uniform depth u independence of sibling value u n n and the pathology disappears. It is difficult to argue convincingly that any of those factors is the cause of pathology. The real virtue of game-tree pathology is to remind us that maximizing of uncertain value is statistically misguided.

Learning Two Player Evaluation Functions n n n We turn our attention to the problem of learning heuristic functions for two-player games. The most obvious, and still most commonly used technique, is hand-coding by a human expert. Example: u Deep_blue(chess) u Chinook’s(checkers).

Samuel’s Checker Player n n n Arthur Samuel’s checkers program, written in the 1950’s. In 1962, running on an IBM 7094, the machine defeated R. W. Nealy, a future Connecticut state checkers champion. One of the first machine learning programs, introducing a number of different learning techniques.

Samuel’s Checker Player n Rote Learning u When a minimax value is computed for a position, that position is stored along with its value. u If the same position is encountered again, the value can simply be returned. u Due to memory constraints, all the generated board position cannot be stored, and Samuel used a set of criteria for determining which positions to actually store.

Samuel’s Checker Player n Learning the evaluation function u Comparing the static evaluation of a node with the backed-up minimax value from a lookahead search. F If the heuristic evaluation function is perfect, the static value of a node would be equal to the backed-up value based on a lookahead search applying the same evaluation on the frontier nodes. F If there’s a difference between the values, the evaluation the heuristic function should be modified.

Samuel’s Checker Player n Selecting term u Samuel’s program could select which terms to actually use, from a library of possible terms. u In addition to material, these terms attempted to measure following board features : F center control F advancement of the pieces F mobility u The program computes the correlation between the values of these different features and the overall evaluation score. If the correlation of a particular feature dropped below a certain level, the feature was replaced by another.

Linear Regression n n Samuel’s method for modifying the weights were somewhat Ad Hoc. We shell describe a more principled way of performing this task.

Linear Regression n Consider a checkers evaluation function based just on material, of the form: cppw+ckkw-cppb-ckkb, where u pw/pb - number of single white/black pieces on the board u kw/kb - number of white/Black kings on the board u cp/ck - coefficient/weight assigned to a single pieces/king n Since the game is symmetric with respect to white and black, we assigned the same pieces the same weights.

Linear Regression n n An individual function is represented by a particular set of values for cp and ck. We represent all such function in a two dimensional space with cp on one axis and ck on the other. n n our task: to find the best point in this space. We start with an initial approximation of the relative weights and find out from the equation the static heuristic value of the current state.

Linear Regression n From this game state, u We perform a lookahead search as deeply as our computational resources allow. u At the frontier, our current evaluation function is applied to the leaf nodes, and these values are minimaxed back up to the root to determine a backed up value b for the position. u In general, this value will not equal the static value of the node.

Linear Regression n Consider the equation : b=cppw+ckkw-cppb-ckkb, n n n It represents the set of all possible weight combination for which the static value of this particular position will equal its backed-up minimax value from the given depth. This is the equation of a line which is based on only a single game state. Each state produce another line.

Linear Regression n n In general these lines will intersect one another, but not all at the same point. Given a set of such lines, we can determine the point which more nearly approximates their mutual intersection. Standard mathematical techniques such as linear regression can be applied to solve this problem. The best intersection correspond to a new point in cpck space , and hence a different evaluation function.

Linear Regression n The entire process used a particular approximation of the evaluation function, which was applied to the leaf nodes of each minimax search. The new function must be viewed as simply a different , and hopefully better, approximation. To get an even better function, we must return the entire process again, applying the new approximation to the leaf nodes to get yet another approximation.

Linear Regression n We have two loops to this process. u the inner loop uses a particular evaluation function to derive a new approximation. u the outer loop iterates this process over multiple approximation. n Hopefully it will eventually converge to a particular function or a small neighborhood of such function.

Experiments with Chess n n As a test of these ideas, was the task of learning the relative weights of the different chess pieces. The evaluation function was based purely on material, with five parameters - the weights of : u queens u rooks u bishops u knights u pawns

Experiments with Chess n n n Initially all weights were set to one. The lookahead search was limited to two levels deep, and linear regression was used to derive each successive approximation to the evaluation function. Surprisingly, the values eventually converged to a fixed point.

Experiments with Chess Result n n These are not the same, but they are close and atlas the order of the pieces is correct. Bear in mind that this experiment was performed with a purely material evaluation function, and only two-level lookahead !