State Agnostic Planning Graphs William Cushing Daniel Bryce

State Agnostic Planning Graphs William Cushing Daniel Bryce Arizona State University {william. cushing, dan. bryce}@asu. edu Special thanks to: Subbarao Kambhampati, David E. Smith, Menkes van den Briel, Romeo Sanchez, J. Benton

Introduction Motivation § Reachability analysis (via Planning Graphs) § Sets of planning graphs are useful § § § Progression search Belief-space planning Replanning Robustification Local search … § …but highly redundant 1 3 o 34 q q 5 p 5 5 opq 5 o G 3 oppr 5 o 56 opq opr 5 o 56 1 p 6 rpo q 5 or 5 p 6 34 56 6 o 12 1 3 oqt oqr oo 56 rq ps orp oqt o 56 o 67 pq opr o 67 5 5 § PGs overlap (duplicate information) § PGs are inflexible (fixed source) § Generalize PG to multiple sources o 56 4 G 3 4 5 6 1 2 5 6 1 o 12 2 3 3 1 o 12 1 o 23 q t r p 5 r q 6 q r ps 5 t p 56 q 6 r 7 6 7 oqso 34 o 45 otp ors o o 67 o G 34 oqt o 45 o 56 ops 67 o 67 oqt o 12 o 78 o 23 o 56 o 67 q t r s pr q 5 p 6 s 7 t p 5 q 6 r 7 s t 6 7 8 2 G o. G 3 4 5 G G o. G 3 4 5 6 7 )=5 h( 1 2 3 5 6 7 o. G G

Introduction Overview State Agnostic Graph Build. SAG() Extract. H(A, B) Reachability Queries Build. PG(A) Planning Graphs Technique: Transform Build. PG(A) into Build. SAG() 1. Labeled Uncertainty Graph [LUG] 2. (Belief) State Agnostic LUG [SALUG] Scratch 3. Optimized (Belief) State Agnostic LUG [SLUG]

Heuristics for belief-space Multiple Graphs 1 3 3 5 1 5 o 12 o 34 o. G o 34 o 56 o 12 o 56 G 1 2 3 4 G 3 4 5 6 1 2 5 6 o 12 o 23 o 34 o 45 o. G o 34 o 45 o 56 o 67 o 12 o 23 o 56 o 67 h( )=5 1 2 3 o. G G 4 5 G 3 o. G G 4 5 6 7 1 2 3 o. G G 5 6 7 D. Bryce and S. Kambhampati, “Heuristic Guidance Measures for Conformant Planning”, In ICAPS’ 04, 2004.

Heuristics for belief-space Unioned Graphs 1 3 1 3 5 1 5 o 12 o. G o 34 o 56 o 12 o 56 G 1 2 G 3 4 1 G 2 3 4 5 6 1 2 5 6 o 12 o 23 G o 34 o 45 o 56 o 67 o 12 o 23 o 56 o 67 1 2 G 3 o. G G 4 15 G 2 3 o. G G 4 5 6 7 1 2 3 5 6 7 o. G G

Heuristics for belief-space Unioned Graphs 1 3 o. G 1 3 5 1 5 3 5 o 12 o 34 o 56 G G 1 2 3 4 5 6 o. G o 12 o 23 o 34 o 45 o 56 o 67 G 1 2 3 4 5 6 7 h( o. G )=1 G

Heuristics for belief-space Labeled Graph [LUG] 1 3 5 3 5 o 12 o 34 o 56 o. G 1 2 3 4 5 6 h( G G o 12 o 23 o 34 o 45 o 56 o 67 )=5 G o. G 1 2 3 4 5 6 7 1 5 D. Bryce, S. Kambhampati, and D. Smith, “Planning in Belief Space with a Labelled Uncertainty Graph”, In AAAI Workshop on Markov Processes, 2004.

Heuristics for belief-space Labeled Graph [LUG] 1 3 5 1 5 G G o 12 3 o 34 5 o 56 1^ 3^ 1^ 1^ 3^ 5^ 1 v 3 o. G 1 2 3 4 5 6 o 12 o 23 o 34 o 45 o 56 o 67 3 ^ -5 ^ γ 5 ^ -1 ^ γ 5 ^ -3 ^ γ 3 v 5 ^ -3 v-5 ^ γ 1 v 5 ^ -1 v-5 ^ γ 1 v 3 ^ -1 v-3 ^ γ ^ 3 v 5 ^ 1 v 5 ^ -1 v-3 v-5 ^ γ γ = “everything else false” G G o. G 1 2 3 4 5 6 7 § Binary Decision Diagrams § Initialize: and/projection § Operator: and/preconditions § Literal: or/supporters D. Bryce, S. Kambhampati, and D. Smith, “Planning in Belief Space with a Labelled Uncertainty Graph”, In AAAI Workshop on Markov Processes, 2004.

Single graph for progression Multiple (Labeled) Graphs G G 1 3 o. G 1 o 12 1 2 3 3 5 o 34 3 4 5 1 o 56 5 o 12 o 23 o 34 o 45 o 56 o 67 o. G 1 1 2 3 1 o 34 3 4 o 12 o 23 3 o 34 4 5 5 5 1 2 3 1 5 o 12 o 56 o 45 5 o 56 6 6 o 67 2 3 1 o 34 3 4 5 o 56 5 6 o 12 o 23 o 34 o 45 o 56 o 67 o. G 1 2 3 4 5 6 7 o. G 1 G o 12 3 G G 5 G G o. G 7 o. G 3 G o. G G G o. G 1 1 2 3 3 4 5 6 7 1 1 2 3 1 5 o 12 o 34 3 4 5 o 56 5 6 o 12 o 23 o 34 o 45 o 56 o 67 1 2 3 4 5 6 7 D. Bryce, S. Kambhampati, and D. Smith, “Planning Graph Heuristics for Belief Space Search”, ASU CSE TR, 2005.

Single graph for progression Unioned (Labeled) Graph G G 1 3 o. G 1 o 12 1 2 3 3 5 5 1 3 o 34 3 4 o 56 5 6 o 23 o 45 o 56 o 67 5 1 1 2 3 o. G 3 1 5 o 12 1 2 o 34 3 4 5 o 56 5 6 o 34 o 45 o 56 o 67 1 G 3 4 o. G 5 o 56 1 o 56 G o. G 1 2 3 3 4 5 6 7 1 1 5 6 4 5 o 12 1 o 34 3 4 5 4 3 2 3 5 o 67 o. G 2 o 56 1 G 3 7 1 o 45 5 6 o 67 o 34 4 2 o 34 o 56 o 23 3 1 o 23 o 45 5 o 12 2 o. G o 12 6 G o 23 o 34 6 o 34 o 12 3 G o. G 2 5 5 o 12 o. G 1 3 4 G 1 5 G 7 3 o. G 3 o 34 G G o 12 1 3 5 G o 56 5 6 G o. G 6 7 o 12 o 23 o 34 o 45 o 56 o 67 1 2 3 4 5 6 7

Single graph for progression Labeled (Labeled) Graph [SALUG] § Introduce labels for beliefs over labels for states G G o. G 1 3 5 3 1 5 o. G 1 G o. G 1 Sr v S o 12 b o 12 Sb v Sg 2 2 Sr v Sg o 23 Sr^Sb^Sg => 1 v 3 ^ 3 v 5 ^ 1 v 5 ^ -1 v-3 v-5 ^ γ 3 -Sr => 5 ^ 1 v 3 3^ -1 v-3 ^ γ o 34 1 ^ 3 v 5 ^ -3 v-5 ^ -Sb => γ 4 -Sg => 3 ^ 1 v 54^ -1 v-5 ^ γ o 45 5 5 o 56 6 6 o 67 7 W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005.

Single graph for progression Labeled (Labeled) Graph [SALUG] G G 1 3 o. G 1 o 12 2 3 5 o. G 1 o 12 2 o 23 3 o 34 4 1 5 G 4 o 45 5 5 o 56 6 6 o 67 7 W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005.

Optimized single graph Filtered Unioned (Labeled) Graph [SLUG] Don’t let the name fool you! G G 1 3 3 5 1 5 o. G 1 3 5 o 12 o 34 o 56 o. G 1 2 3 4 5 6 o 12 o 23 o 34 o 45 o 56 o 67 G o. G 1 2 3 4 5 6 7 § Ignore irrelevant labels Ø Largest LUG == all LUGs W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005.

Empirical Results Belief Space Problems Conformant Classical Problems Contingent

Conclusion § Developed general agnosticism (SAG) § Removed dependence on world state (PG LUG) § Removed dependence on belief state (LUG SALUG) § Dramatically improved performance ({LUG, SALUG} ~> SLUG) § Empirically demonstrated § Internal performance boost § Favorable external comparison § SAG has rich connections to: § Constraint propagation (vs. branching) § Lazy evaluation § Memoization

Further Details § Heuristics for belief space in the CAlt. Alt planner § Labeled Uncertainty Graph in the CAlt. Alt planner § Heuristics and LUG in the POND and CAlt. Alt planners § SLUG: Improvement to LUG for POND § CLUG: propagating numeric information § D. Bryce and S. Kambhampati, “Heuristic Guidance Measures for Conformant Planning”, In ICAPS’ 04, 2004. § D. Bryce, S. Kambhampati, and D. Smith, “Planning in Belief Space with a Labelled Uncertainty Graph”, In AAAI Workshop on Markov Processes, 2004. § D. Bryce, S. Kambhampati, and D. Smith, “Planning Graph Heuristics for Belief Space Search”, ASU CSE TR, 2005. § W. Cushing and D. Bryce, “State Agnostic Planning Graphs”, In AAAI, 2005. § D. Bryce and S. Kambhampati, “Cost Sensitive Reachability Heuristics for Handling State Uncertainty”, In UAI’ 05, 2005.