Advances of ANDORtree Search Algorithms in Shogi Mating

Advances of AND/OR-tree Search Algorithms in Shogi Mating Search Hiroyuki Iida　and Makoto Sakuta Department of Computer Science Shizuoka University, Japan The 6 th Computer Olympiad: Computer-Games Workshop (Maastricht, 2001)

Overview • Tsume-shogi (mating search) problem is a good application for AND/OR-tree search algorithms • Algorithms for solving tsume-shogi – ID, AO*, PN search, PN*, PDS and DF-PN – Equivalent relations between these algorithms • Tsume-shogi in actual shogi endgame – Find a relatively long-step winning sequence

Main rules of tsume-shogi • Two agents: attacker and defender • Attacker has to continue the check move sequences. • Attacker must mate in the shortest sequence. • Defender must select a move which makes the mating sequence as long as possible. • The length of the solution sequence is not given in advance. • See also [Seo, Iida and Uiterwijk, 2001]

Why tsume-shogi is a good application for AND/OR-tree search? • Tsume-shogi has an enormous search space, so simple brute-force search is not adequate. • A tsume-shogi problem has a unique solution. • There are various types of problems, including those very hard to solve and those having a very long solution sequence. • Many high-performance tsume-shogi solving programs have been developed, so the performance of a program can be measured and compared.

Examples of solving tsume-shogi　(1) • An endgame position from GM’s game. (mate in 17 plies, relatively not so long) – ID solved in 22. 6 seconds. – PN solved in 7. 93 seconds. – PDS solved in 6. 11 seconds. – PN* solved in 4. 87 seconds. • See the detail (machine spec, size of transposition table, etc. ) in [Sakuta and Iida, ACG 9].

Examples of solving tsume-shogi (2) • An endgame position from GM’s game. (mate in 37 plies, relatively long in actual games) – ID was unable to solve. – PN* solved in 1000 seconds. – PDS solved in 757 seconds. • See the detail (machine spec, size of transposition table, etc. ) in [Sakuta and Iida, ACG 9].

Why proof-number search? • Best-first search algorithms such as proof-number (PN) search work well when solving endgame positions in chess -like games. (also other games: GO, LOA, etc. ) • However, it is hard, even for PN search, to solve an endgame problem with very long sequences because of its inherent memory limits. • Recently, innovative depth-first search algorithms based on proof/disproof numbers were developed (PN*, PDS and DF-PN) for solving tsume-shogi problems, among which one with 1525 steps. • These depth-first algorithms behave like best-first search.

AND/OR-tree search • Best-first search AO* (Nilsson) and its variants A* for OR Graph Minimax tree CN-search (Mc. Allester, 1988) (Schaeffer, 1990) conspiracy number PN-search (Allis, 1994) proof number disproof number

PN-search (Allis, 1994) • PN (proof number) search is a best-first AND/OR tree-search algorithm. • Inspired by conspiracy-number search algorithm (Mc. Allester, 1988). • Before starting the search, a search goal is defined. E. g. , try to mate. The evaluation of a node returns one of three values: true, false, or unknown. The evaluation is seen from the point of view of the player to move in the root position. The value true indicates that the player to move in the root position can achieve the goal, while false indicates that the goal is unreachable. • A node is proved if its value has been established to be true, whereas the node is disproved if its value has been determined to be false. • A node is solved as soon as it has been proved or disproved. • A tree is solved (proved or disprove) if its root is solved. • The goal of PN search is to solve a tree.

PN-search (continue) • Has high ability for solving AND/OR trees. • However, it expands the whole search tree in working memory and require much memory. • This means that there is a severe memory restriction for hard problems. • The detail of PN search [Allis et al. , 1994] • PN^2 search (Breuker et al. , 1999) • Would it be possible to transform a best-first PN search into a depth-first search?

PN* (Seo, 1995) • is recursive iterative-deepening depth-first search algorithm using proof numbers as a criterion to develop the frontier nodes. • PN* starts searching by setting the proof-number threshold to 1 and expands the root node. If the proof number of the node under consideration exceeds the threshold, PN* stops node expansion at this node. When all frontier nodes have proof numbers exceeding the proof-number threshold, the search tree is discarded, the threshold is incremented and a new search starts. • This iterative searching is enhanced by storing the expanded nodes and their properties in the transposition table.

PN* (continue) • As the threshold is getting larger, node expansion proceeds in such a way that nodes having relatively small proof numbers are expanded first, like in bestfirst search, without the disadvantage of the memory requirement. • PN* uses the method of recursive iterative deepening. At an AND node threshold values assigned to the child OR nodes are iteratively increased from 1 to their current maximum. • Enhanced by dynamic evaluation, efficient successor ordering and pruning by dependency relations. • The detail of PN* [Seo, Iida and Uiterwijk, 2001]

Recursive iterative deepening at root node best-first search (e. g. pn-search) threshold=2 frontier nodes threshold=3 threshold=4 recursive iterative deepening iterative-deepening depth-first search depth=1 threshold=2 depth=2 or 3 threshold=4 depth=3 or 5 depth=4 or 7 depth=5 or 9

Equivalence (1): AO* and PN*　(Nagai, 1999) • PN* (depth-first search) algorithm behaves the same as AO* (best-first search) in the meaning that PN* always expands a most-proving node. • Most proving node (in the context of AO*) is a leaf node selected by tracing from the root in the following way. • For each OR node, trace the child with minimum proof number. • For each AND node, trace any child with non-zero proof number.

Tsume-shogi solver: SEO • based on the PN* algorithm • excellent management of transposition table • uses Dominance Relation of positions • omits the useless moves of dropping pieces • other domain-specific features • solved “Microcosmos”, the problem with the longest steps, i. e. , 1525 plies (1997)

PDS (Nagai, 1998) • PDS (Proof-number and Disproof-number Search) is a straight extension of PN*, using both proof numbers and disproof numbers. PDS uses two thresholds in searching, the proof-number threshold and the disproof-number threshold. • If the proof number or the disproof number exceeds the threshold at a certain node, PDS stops node expansion at this node. When PDS fails the root node expansion, it increases the either threshold value and restarts the searching. • Note that the basic concept of PDS differs from PN search, i. e. , asymptotically equivalent to PN search.

DF-PN (Nagai, 1999) • Depth-first version of PN-search • DF-PN first sets, at the root node, the thresholds of both proof number and disproof number to a certain large value (∞). Then, the threshold values are distributed among the descendant nodes. • At every node, recursive iterative deepening is performed (cf. only OR nodes in PN*)

Equivalence (2): PN and DF-PN (Nagai, 1999) • DF-PN (depth-first search) algorithm behaves the same as PN (best-first search) in the meaning that DF-PN always expands a most-proving node. • Most proving node (in the context of PN) is a leaf node selected by tracing from the root in the following way. • For each OR node, trace the child with minimum proof number. • For each AND node, trace the child with minimum disproof number.

Tsume-shogi solver: NAGAI • based on the DF-PN algorithm • efficient use of transposition table – Small. Tree. GC and Small. Tree. Replacement (Nagai, 99) • pruning by dependency relations (positions and proof/disproof numbers) – More efficient than the solver SEO • GHI problem (cf. Breuker et al. , CG 98) • Proof pieces (Seo, 99) and disproof pieces • detecting a DAG to avoid the double counting of proof numbers

Performance of PN*, PDS and PN search (Sakuta and Iida, ACG 9, 1999) • Experiments in two domains: – tsume-shogi (derived from GM’s games), and – 6 x 6 othello (positions on PV) • Comparison shows that – In both, PDS outperforms PN*. – In othello, PN search variants do not show remarkable advantage over depth-first search. – In shogi, PN search variants show remarkable advantage over depth-first search. • Forcing move sequences and sudden termination. • Disproving ability is important as well as proving.

Disproving ability compared in tsume-shogi

Four algorithms (equivalent)

More detail comparison Iterative deepening PN PN* PDS DF-PN behaviour depth-first best-first depth-first transposition table yes basically no indispensable conspiracy number no proof & disproof & proof & threshold disproof iterative deepening depth no recursive iterative deepening no no at OR nodes at AND/OR nodes iteration at the root node yes proof & no threshold disproof

Advantages of depth-first-type algorithm • If a best-first algorithm and a depth-first algorithm behave in the same way, it is certainly better to adopt the depth-first one. – IDA* and A*, MT-SSS* and SSS*, and – MTD(f) for minimax trees [Plaat et al. , 1996] • Many enhancements can be used: – History heuristics, transposition table, iterative deepening, and so on. • Depth-first algorithm uses less memory space – Search speed is often faster in practice

Summary of experimental performance • PN* solved most hard problems, among which one with 1525 steps. But, it was unable to solve 10 hard problems with long steps over 300. • PDS basically outperforms PN* in solving long-step problems. • DF-PN have solved all hard-problems.

Application to actual endgames in shogi • Advances of AND/OR tree search algorithms enable a computer to: – Find a mate under tournament time constraints, at interior nodes (close to the root). – Reuse the outcome of mating search in the successive searches. – Find a brink-mate (forcing mate threat) – Play a high-level endgame (mating race).

Conclusions • Depth-first searches using proof/disproof numbers have recently been developed. – Basically depth-first search but behaves in best-first manner like proof number search (PN search). – Memory requirement is much less than best-first. • DF-PN have solved all hard-problems of tsumeshogi with long steps over 300. • These algorithms are now used in shogi playing programs. – High-level mating race in the endgame.