From ANDOR Search to ANDOR BDDs Rina Dechter
From AND/OR Search to AND/OR BDDs Rina Dechter Information and Computer Science, UC-Irvine, and Radcliffe Institue of Advanced Study, Cambridge Joint work with Robert Mateescu and John Cobb Verification/constraints workshop, 2006
Combining Decision Diagrams f = A*E + B*F A B g = C*G +D*H C B D E G F G H 0 1 A B B C D E OBDD f * g = F C D E F D E E F G G H 0 D 1 E
Combining AND/OR BDDs f = A*E + B*F A B g = C*G +D*H C B D E G F G H 0 1 0 AND A B B AOBDD f * g = E C D D E G F 0 1 G H 1 0 1
OBDDs vs AOBDD AND A AOBDD B C B E f *g= D D E G F G H 0 1 A B B OBDD C D E F D D E E F G G H 0 1 E
Outline n n n Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR graphs to AOMDDs Compilation of AOMDDs and earlier BDDs
Constraint Networks A Example: map coloring Variables - countries (A, B, C, etc. ) Values - colors (red, green, blue) Constraints: Constraint graph A B red green yellow red yellow green red A E D B F C Semantics: set of all solutions Primary task: find a solution D B F G G C
Propositional Satisfiability = {(¬C), (A v B v C), (¬A v B v E), (¬B v C v D)}.
Graphical models n A graphical model (X, D, C): n n n X = {X 1, …Xn} variables D = {D 1, … Dn} domains C = {F 1, …, Ft} functions (constraints, CPTS, cnfs) A F B C Examples: • Constraint networks • Belief networks • Cost networks • Markov random fields • Influence diagrams E n n n D All these tasks are NP-hard identify special cases approximate
Tree-solving is easy CSP – consistency (projection-join) Belief updating (sum-prod) P(X) P(Y|X) P(T|Y) MPE (max-prod) P(Z|X) P(R|Y) P(L|Z) P(M|Z) #CSP (sum-prod) Trees are processed in linear time and memory
Transforming into a Tree n By Inference (thinking) n n Transform into a single, equivalent tree of sub-problems By Conditioning (guessing) n Transform into many tree-like subproblems.
Inference and Treewidth ABC G D A DGF B BDEF F C E H M K L EFH FHK J HJ KLM Inference algorithm: Time: exp(tree-width) Space: exp(tree-width) treewidth = 4 - 1 = 3 treewidth = (maximum cluster size) - 1
Conditioning and Cycle cutset H G D F O E C M J O E C P B K A K L Cycle cutset = {A, B, C} H F P L C N D F O E H G M J K B N D P B L G N D F M A J H G N M J O E C P K L
Search over the Cutset E Graph Coloring problem M D • Inference may require too much memory L C A B • Condition (guessing) on some of the variables K H F G J A=yellow E D A=green M E L C B K H F G J D M L C B K H F G J
Search over the Cutset (cont) E Graph Coloring problem M D • Inference may require too much memory L C A B • Condition on some of the variables K H F G J A=yellow B=red E D B=blue M E L C D B=green M H G J E L C K F A=green D M H G J E L C K F B=red D B=blue M L C K H F G J E D B=yellow M L C K H F G J E D M L C K H F G J
Inference vs. Conditioning n By Inference (thinking) A C B ABC G D BDEF F E H J L FHK KLM Variable-elimination Directional resolution Join-tree clustering By Conditioning (guessing) A=yellow E D M L C A B K H F G J Exponential in treewidth Time and memory EFH M K HJ n DGF A=green B=blue B=green B=red B=blue E D M L C L C K K H H F F G J G J Exponential in cycle-cutset Time-wise, linear memory Backtracking Branch and bound Depth-first search
Outline n n n Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR graphs to AOMDDs Compilation of AOMDDs and earlier BDDs
Classic OR Search Space A F B E A 1 0 E 1 0 C F D 0 B D Ordering: A B E C D F C 1 0 0 1 1 0 0 1 0 1 1 0 0 0 1 1 1 0 1 0 1 0 1 0 1 0 1 1 0 0 0 1 1 0 1 0 1 1 0 1 0 1 0 1 0 1 0 1
AND/OR Search Space A A A F B C B C C E E D E Primal graph D D F DFS tree OR A AND 0 1 OR B B AND 0 OR AND E 0 OR AND C 1 E 0 1 D 0 F 1 0 0 D 1 0 F 1 0 0 0 F 1 0 1 D 1 E C 1 0 D 1 1 0 1 0 E 0 1 D F 1 C F 1 0 0 D 1 0 1 F 1 0 C 0 1 D 1 0 F 1 0 1
A A F B AND/OR vs. OR OR B C E E C D D F A AND 0 1 OR B B AND 0 OR AND 0 C 1 OR 0 E 0 1 D AND F 1 0 1 E 0 0 D 1 0 F 1 0 0 1 0 F 1 0 D 1 0 C 0 1 D 1 E C 1 AND/OR 1 1 0 0 1 D F 0 1 E F 1 0 0 D 1 0 1 F 1 0 C 0 1 D 1 F 0 1 0 D 1 0 F 1 0 1 AND/OR size: exp(4), OR size exp(6) A 0 B 0 E F 1 0 C D 1 1 0 0 0 1 1 0 0 1 0 1 1 0 0 0 1 1 0 1 0 1 0 1 0 1 0 1 1 0 0 0 1 OR 1 0 1 0 1 0 1 0 1 0 1 0 1
AND/OR vs. OR with Constraints OR F B E C D D F A 0 1 OR B B AND 0 OR 0 C 1 OR 1 D AND 0 E 0 F 1 0 0 D 1 0 F 1 0 1 A 0 0 F 1 0 D 1 0 C 0 1 D 1 E C 1 AND/OR 0 1 E AND 1 1 0 0 1 D F 0 1 E F 1 0 0 D 1 0 1 0 C 1 0 0 1 1 0 1 0 1 1 D 1 F 0 1 0 D 1 0 0 0 F 1 0 1 1 0 E C 1 F 0 B F B C E AND D A A No-goods (A=1, B=1) (B=0, C=0) 0 1 1 0 0 0 1 1 0 1 0 1 0 1 0 1 0 1 1 0 0 0 1 OR 1 0 1 0 1 0 1 0 1 0 1 0 1
AND/OR vs. OR with Constraints OR F B 1 OR B B AND 0 OR E 0 C E 1 1 0 D AND 0 1 0 0 0 F 1 0 1 D 1 E C 1 F A C D F AND/OR 0 D 1 0 C 1 1 D F 1 0 0 1 F 1 0 B 1 0 0 1 1 C F E 0 1 OR D D B A 0 E C E AND A A No-goods (A=1, B=1) (B=0, C=0 0 1 1 0 1 0 1 0 1 1 0 0 0 1 OR 0 0 1 1 1 0 1 0 1 0 1 1 0 1 0 1 0 1
DFS algorithm (#CSP example) A A F B B C solution E E C D D OR F 11 A AND 0 5 1 6 OR B 5 B 6 AND 0 OR AND 0 1 0 D 2 2 F 1 D 1 0 4 1 2 F 0 D 1 F 1 D 1 F 0 0 0 1 0 1 1 1 1 0 0 1 0 0 D 0 1 1 F 1 D 2 F 1 0 1 0 1 1 1 0 Value of node = number of solutions below it 0 1 1 D 0 1 2 C 1 0 2 2 1 E 1 C 4 E 1 Marginalization C 1 E OR node: operator (summation) 1 0 0 1 operator 0 2 1 2 0 1 node: 0 1 1 0 0 AND Combination (product) 1 0 2 C 1 OR AND 1 2 E 1 4 1 1 1 0 1 1 F 0 1 1 0 1 D 0 0 1 1 F 1 0 1 1 1 0
Pseudo-Trees (Freuder 85, Bayardo 95, Bodlaender and Gilbert, 91) 4 1 1 6 2 3 m <= w* log n 2 7 5 (a) Graph 4 1 2 3 7 1 7 3 5 4 5 3 4 2 7 6 (b) DFS tree depth=3 (c) pseudo- tree depth=2 5 6 6 (d) Chain depth=6
Tasks and value of nodes n V( n) is the value of the tree T(n) for the task: n n n Consistency: v(n) is 0 if T(n) inconsistent, 1 othewise. Counting: v(n) is number of solutions in T(n) Optimization: v(n) is the optimal solution in T(n) Belief updating: v(n), probability of evidence in T(n). Partition function: v(n) is the total probability in T(n). n Goal: compute the value of the root node recursively using dfs n AND/OR search tree and algorithms are search of the AND/OR tree. ([Freuder & Quinn 85], [Collin, Dechter & Katz 91], [Bayardo & Miranker 95], [Darwiche 2001], [Bacchus et. Al, 2003]) n Space: O(n) n Time: O(exp(m)), where m is the depth of the pseudo-tree n Time: O(exp(w* log n)) n BFS is time and space O(exp(w* log n)
F From AND/OR Tree B B C E K C H A AND 0 1 OR B B AND 0 OR AND E D G OR A J A E 0 C 1 OR 0 1 E 0 1 D F 0 D 1 F 0 0 1 D F D C 1 G J H K E 0 1 D F F 1 E C D F C 0 D 1 F 0 1 D F AND 0 1 0 1 0 1 0 1 OR G G J J G G J J AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 OR H H H H K K K K H H H H K K K K AND 0101010101010101010101010101010101010101010101010101010101010101
To an AND/OR Graph F B B C E K A 0 1 OR B B AND 0 OR OR C H AND E D G OR A J A E 0 C 1 D 0 1 E 0 1 F D 0 F 1 D 0 0 1 F D C 1 D F AND 0 1 OR G G J J AND 0 1 0 1 OR H H K K AND 01010101 F G J H K 1 E C D E 0 1 F D 0 F C 1 0 D 1 F D F
AND/OR Search Graphs n n n Any two nodes that root identical subtrees/subgraphs (are unifiable) can be merged Minimal AND/OR search graph: of R relative to tree T is the closure under merge of the AND/OR search tree of R relative to T, where inconsistent subtrees are pruned. Canonicity: The minimal AND/OR graph relative to T is unique for all constraints equivalent to R.
Context based caching Caching is possible when context is the same n context graph = n = parent-separator set in induced pseudo- current variable + ancestors connected to subtree below A J A F B C E D G H context(B) = {A, B} B K E C D F G J H K context(C) = {A, B, C} context(D) = {D} context(F) = {F}
Caching F B AND OR 0 E 0 1 D F F G J H K B C 1 D 1 1 E C A 0 OR E H B AND K D G context(B) = {A, B} context(C) = {A, B, C} context(D) = {D} 0 context(F) = {F} B C E OR A J A 0 D F E C 1 0 0 1 D 1 F D C 1 0 1 D F E F C 0 D 1 F 0 1 D F AND 0 1 0 1 0 1 0 1 OR G G J J G G J J AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 OR H H H H K K K K H H H H K K K K AND 0101010101010101010101010101010101010101010101010101010101010101
A J A F Caching B E context(D)={D} K A 0 1 OR B B 0 OR AND OR 0 C 1 D 0 1 E E 0 1 F D 0 F C H AND E D G context(F)={F} OR B C 1 D 0 0 1 F D C 1 D F AND 0 1 OR G G J J AND 0 1 0 1 OR H H K K AND 01010101 F G J H K 1 E C D E 0 1 F D 0 F C 1 0 D 1 F D F
C A D F B E All Four Search Spaces A B C D E F 0 0 1 1 0 0 0 1 A 1 0 1 1 0 0 1 0 1 B 1 0 0 1 C 1 0 0 D 1 0 1 0 01 01 0 1 0 1 01 010 1 0 1 01 0101 01 0 0 1 E 0 1 0 1 1 1 0 0 1 1 0 0 F 0 1 1 1 Full OR search tree Context minimal OR search graph 126 nodes 28 nodes OR 1 A AND 0 1 OR B B AND 1 0 C OR C E 1 0 C E E AND 0 1 0 1 OR AND D D F F 0 1 0 1 0 1 Full AND/OR search tree 54 AND nodes OR AND A 0 1 B B 1 0 C C E 1 0 C E 0 1 0 D D F F 01 D C E E 1 0 1 D F F 01 0 1 0 1 Context minimal AND/OR search graph 18 AND nodes
Properties of minimal AND/OR graphs Complexity: n Minimal AND/OR R relative to pseudo-tree T is O(exp(w*)) where w* is the tree-width of R along T. n Minimal OR search graph is O(exp(pw*)) where pw* is pathwidth n w* ≤ pw*, pw* ≤ w*log n Canonicity: n The minimal AND/OR search graph is unique (canonical) for all equivalent formulas (Boolean or Constraints), consistent with its pseudo tree.
Searching AND/OR Graphs n AO(j): searches depth-first, cache i-context n j = the max size of a cache table (i. e. number of variables in a context) i=0 i=w* j Space: O(n) Space: O(exp w*) Time: O(exp(w* log n)) Space: O(exp(j) ) Time: O(exp(m_j+j ) O(exp w*)
Context-Based Caching A B C E D F context(A) = {A} context(B) = {B, A} context(C) = {C, B} context(D) = {D} context(E) = {E, A} context(F) = {F} C D 3 B 0 0 1 1 C Value 0 5 1 2 0 2 1 0 C E 0 7 1 B 7 B 5 0 4 E 5 0 D Space: O(exp(2)) B A 5 6 F Primal graph 5 Cache Table (C) A 2 C 1 D 6 1 E 4 4 0 C E 2 0 D 0 1 D 4 C 1 E
Outline n n n Background in Graphical models AND/OR search trees and Graphs Minimal AND/OR graphs From AND/OR search graphs to AOMDDs Compilation of AOMDDs and earlier BDDs
OR Search Graphs vs OBDDs A A 0 0 1 1 B 0 0 1 1 C 0 1 0 1 f(ABC) 0 0 0 1 0 1 0 A B B 0 0 C 1 0 1 Full AND/OR search tree A A 0 1 1 0 C B 1 1 0 1 1 1 Minimal AND/OR search graph Backtrack free AND/OR search tree redundant 1 B 1 C C C 1 0 1 B B 1 0 C C B 1 A 0 1 1 An OBDD
AND/OR Search Graphs; AOBDDs A F(A, B, C, D)= (0, 1, 1, 1), (1, 0, 1, 0), (1, 1, 1, 0) B C D AOBDD(F) A A 0 0 1 1 B redundancy B B 0 1 1 1 D C D 1 1 0
AND/OR Search Graphs; AOBDDs A F(A, B, C, D)= (0, 1, 1, 1), (1, 0, 1, 0), (1, 1, 1, 0) B C D AOBDD(F) A A 0 0 1 1 B redundancy B B 0 1 1 1 D C D 1 1 0
AOBDD Conventions A B D C A A 0 0 Point dead-ends to terminal 0 Point goods to terminal 1 1 1 B 0 1 B C 1 0 C D 1 0 1 D 1 0 0 1
AOBDD Conventions A B D C Introduce Meta-nodes A A 0 0 1 1 B 0 B 1 1 C 0 C D 1 0 1 D 1 0 0 1
Combining AOBDD (apply) A 0 0 1 1 B 0 0 1 1 C 0 1 0 1 f(ABC) 0 0 0 1 0 1 A B C A 0 0 1 1 B 0 0 1 1 D 0 1 0 1 A 0 1 0 * 1 C 0 0 A A B B D D C A B 0 g(ABC) 0 0 0 1 1 A 1 0 = B 0 1 D 0 0 D 1 0 B 0 1 C 0 1 1 1 0 D 1 0 1 1
Example: (f+h) * (a+!h) * (a#b#g) * (f+g) * (b+f) * (a+e) * (c#d) * (b+c) A m 7 A D m 7 A A 0 0 1 B 0 1 0 C 1 0 0 1 0 F 1 0 1 F 1 0 B 1 D D G E 1 0 0 1 G 1 0 H 1 0 C H 1 0 0 1 m 6 E G 0 C C 0 1 0 1 C 9 1 0 m 5 1 1 0 1 m 4 D D C D 0 0 C 8 1 1 D 0 E C 1 1 0 m 2 0 1 1 B F 1 0 H G 1 0 C 5 0 1 A A 0 0 0 1 H 1 0 F 1 G 1 m 1 0 1 0 1 C 6 C 7 0 C 0 B 1 0 0 G G 0 E C 3 G 1 0 0 1 1 F 0 1 H 1 0 1 C 4 A 1 0 H B G 1 H m 1 A F 1 B E 1 G 0 1 F F 1 0 m 2 E E G A A 1 0 0 1 m 4 0 0 1 F D A 1 1 B C C 0 0 F 0 m 5 0 1 1 0 B 1 B E D 0 0 C D E pseudo tree 1 B A 1 C 0 D A A 1 H m 3 0 B A F H m 6 0 E C F 1 m 3 F 1 D 0 B primal graph B 0 C B A B F C 1 1 B B G 0 1 H 1 0 1 F 0 1 G C 1 C 2 A F H H
Example (continued) A 0 1 B 0 C 0 B 1 0 C 1 0 F 1 0 1 F 1 0 1 A D 0 D 1 0 E 1 0 G 1 0 H 1 0 1 1 m 7 A A 0 0 1 B 0 1 0 C 1 0 0 1 C 1 0 0 F 1 A B 1 F C 1 1 B B 0 1 F 1 0 B 1 B D 0 D G E 1 0 0 1 G 1 0 H 1 0 C H 1 0 D 0 m 6 1 0 m 3 1 F 1 E G H
AOBDD vs. OBDD D A G C B F E A H B C primal graph A 0 C 0 D B 1 0 C 1 0 D D 1 0 F 1 E 1 0 0 F 1 0 G 1 0 0 H 1 0 1 AOBDD B C 18 nonterminals F G H 1 0 1 G D D 47 arcs E F G G A H H C 1 E F OBDD 27 nonterminals G H D F D A E D C E F 1 B D C D E 1 G 0 C 1 B 0 B H 54 arcs 0
Complexity n n Complexity of apply: Complexity of apply is bounded quadratically by the product of input AOBDDs restricted to each branch in the output pseudo-tree. Complexity of VE-AOBDD: is exponential in the tree-width along the pseudo-tree.
AOMDDs and tree-BDDs n Tree-BDDs (Mc. Millan 1994) are : n AND/OR BDDS are
Related work n Related work in Search n n n Related to compilation schemes: n n Backjumping + learning (Freuder 1988, Dechter 1990, Bayardo and Mirankar 1996) Recursive-conditioning (Darwiche 2001) Value elimination (Bacchus et. Al. 2003) Search over tree-decompositions (Trrioux et. Al. 2002) Minimal AND/OR – related to tree-OBDDs (Mc. Millan 94), d-DNNF (Darwiche et. Al. 2002) Case-factor diagrams (Mcallester, Collins, Pereira, 2005) Tutorial references: Dechter, Constraint processing, Morgan Kauffman, 2003 n Dechter, Tractable Structures of Constraint Satisfaction problems, In Handbook of constraint solving, forthcoming. n
Conclusion n AND/OR search should always be used. n AND/OR BDDs are superior to OBDDs. n A search algorithm with good and no-good learning generates an OBDD or an AND/OR Bdds. n Dynamic variable ordering can be incorporated n With caching, AND/OR search is similar to inference (variableelimination) n The real tradeoff should be rephrased: n n Time vs space rather than search vs inference, or search vs model-checking. Search methods are more sensitive to this tradeoff.
- Slides: 48