Decentralized Mission Planning for Heterogeneous HumanRobot Teams Sameera

Decentralized Mission Planning for Heterogeneous Human-Robot Teams Sameera Ponda Prof. Jonathan How Department of Aeronautics and Astronautics Massachusetts Institute of Technology

Motivation • Modern day complex missions involve networked teams of heterogeneous agents executing several tasks simultaneously: – Unmanned aerial vehicles (UAVs) – target tracking, surveillance – Human operators – classify targets, monitor status – Ground convoys – rescue operations • Key Research Questions: – How can we coordinate team behavior to improve mission performance? – How should planning strategies evolve as we acquire more information? 2

Problem Statement • Goal: Automate task allocation to improve mission performance – Spatial and temporal synchronization – Reduce costs and improve efficiency • Key Technical Challenges: – – – Combinatorial decision problem – computationally intractable (NP-hard) Complex agent modeling & constraints (stochastic, non-linear, time-varying) Limited resources (bandwidth, fuel, etc) Task 3 Task 1 Dynamic networks and communication constraints Task 2 Unknown and dynamic environments Task 4 Agent 5 Agent 1 Agent 4 Agent 6 Agent 2 Task 6 Task 9 Task 5 Agent 3 Task 8 Task 7 3

Planning Approaches • Optimal solution methods are computationally intractable for large problems – Typically use approximation methods Agent 2 Agent 1 • Centralized Planning approach – Mission Control Center (MCC) plans & distributes tasks for all agents – High bandwidth, slow reaction, resource intensive MCC Agent 3 Agent 4 – Individual agents make their own plans and coordinate with each other – Faster reaction to local information changes Agent 1 – Trade-off between communication and computation Agent 2 • Recent research in Decentralized Planning • Key Questions: – What quantities should the agents agree upon? Agent 3 • Information / tasks & plans / objectives / constraints – How do we ensure that the planning is robust to inaccurate information and models? Agent 4 4

Consensus-Based Bundle Algorithm • Decentralized task allocation approach called Consensus-Based Bundle Algorithm (CBBA) [Choi, Brunet, How 2009] – CBBA iterates between 2 phases: Bidding & Consensus Phase 1: Build Bundle & Bid on Tasks (individual agents) 1 2 3 Phase 2: Consensus (all agents) All agents consistent? Yes N No • Core features of CBBA: – Polynomial-time decentralized algorithm with provably good approximate solutions – Consensus on task assignments, not information – guaranteed real-time convergence even with inconsistent information Key extensions to CBBA: 1) Temporal constraints – Time-windows of validity for tasks 2) Connectivity issues and constraints 3) Planning for teams with Humans-in-the-loop 5

CBBA with Time-Windows e. g. monitor status, security shifts Arrival Time-critical e. g. rescue ops, target tracking Score Flat Score • In realistic missions, task scores often depend on arrival times and have associated time-windows of validity: Peak-time Arrival Time e. g. rendezvous, special ops Arrival Time • Issue: Planning algorithms usually involve time discretization – Extra planning dimension – computationally intractable! • CBBA extended to include time-windows – Solution does not discretize time! – Preserves convergence properties – Planner decides arrival times, producing task schedules for agents • Embedded CBBA with Time-Windows into a real-time system architecture 6

CBBA with Time-Windows • CBBA successfully used in real-time fight test environments – Cooperative search, acquisition, and track (CSAT) – Coordination of agents under dynamic network topologies • Further information available online at: http: //acl. mit. edu/projects/cbba. html 7

Connectivity: Network Challenges • As agents move around in the environment, expect varying network topologies – Limited communication radius between agents – Potential broken comm links and/or disconnected networks Task 1 Task 3 Task 2 Agent 1 Agent 2 Agent 5 Disconnected Network Task 4 Agent 4 Task 9 Agent 6 Agent 3 Task 7 Task 6 Task 5 Task 8 • Main issue: Planner cannot converge with a disconnected network, leading to conflicting assignments • Developed two solution approaches: – CBBA with Relays – Creates relay tasks to ensure connectivity – CBBA with Network Handling Protocols – Protocols to adjust task lists prior to planning 8

Connectivity: CBBA with Relays • Extended CBBA to include relay tasks – (Published in Globe. Comm 2010) – Employs underutilized agents as relays – Key feature: Agents use bid info to predict network structure at select times – Guarantees connectivity – Computationally efficient - converges in real-time Relay Task • CBBA with Relays improves team performance and network connectivity 9

Connectivity: Network Protocols • If preventing disconnects is too conservative: Network Handling Protocols to adjust task lists for agents prior to planning – (Published in ACC 2010) Baseline Central (no adjustment) Adjustment Local Adjustment All tasks available to all agents MCC distributes new tasks to closest agents (once per task) MCC distributes tasks to networks at each replan Low Bandwidth High Bandwidth Low Computation High Computation • Local Adjustment improves mission performance Conflicting Guaranteed Assignments Deconfliction –– lower missionscores and with low bandwidth and higher computation requirements wasted lower fuel consumption 10

Planning for Human-Robot Teams • Most modern missions involve human-robot teams – Human operators perform several tasks (e. g. supervisory, target classification, monitoring) – Need to coordinate robotic agents and operators Predator UAV Operations – Associated Press • Main Issue: Operator performance is stochastic – Heterogeneous operator capabilities (“slow” vs. “fast”) – Robustness to uncertainty in team performance • Recent research has explored modeling operators using probabilistic distributions – [Cummings et al ‘ 10] Log-Normal Distribution for Operator Target Identification Figure from [D. Southern, Masters Thesis, 2010] • Key Challenge: Incorporate uncertainty into planner to increase robustness 11

Planning for Human-Robot Teams • Consider a time-critical mission with operators performing target classification • As expected vs. actual service times differ, planner performance degrades – Adding a margin of conservatism can mitigate this problem – Tradeoff between late penalties and number of tasks assigned Agent Schedule • Simulation Observations: Late! Tasks – Performance is best when expected & actual are close (ridge line) – Steeper drop for overestimating (optimistic) vs. underestimating (conservative) Time – Conservative Planning performs better than Optimistic Planning • Developing a Robust Planning Framework – Explicitly embed PDFs of plan parameters – Adapt as estimates improve 12

Planning for Human-Robot Teams • Currently performing Human-in-the-loop experiments at Cornell – CBBA used to allocate targets to agents (MIT) – Image processing and sensor fusion used to update target PDFs (Cornell) – Human-in-the-loop for target classification and PDF updates through HRI (Cornell) 13

Conclusions • Explored strategies to coordinate team behavior to improve mission performance • Extended the Consensus-Based Bundle Algorithm (CBBA) to address the demands of more realistic multi-agent mission planning – Included task time-windows of validity – Addressed connectivity issues and communication constraints – Explored planning for heterogeneous human-robot teams • Current research and expected thesis contributions: 1) Robust decentralized planning framework • Embed distributions of parameters into planner • Preserve computational tractability and scalability (e. g. avoid discretization, explore efficient sampling techniques) 2) Flexible planner structure that adapts to dynamic uncertainty representations • Modular uncertainty representations (Nonparametric Bayesian models, etc) • Modify planning strategy without recomputing all scenarios 3) Efficient strategies for information consensus to improve planner performance • Decide what information and when to share (e. g. hyperparameter consensus) • Cooperative decentralized strategies to update global distributions 14