Intelligent Autonomy Update Marc Steinberg Office of Naval

Intelligent Autonomy Update Marc Steinberg Office of Naval Research (703) 696 – 5115, marc_steinberg@onr. navy. mil Naval Air Systems Command (301) 342 – 8567, marc. steinberg@navy. mil SAE Control & Guidance Committee Meeting, 2 March, 2005

Intelligent Autonomy Future Vision External C 4 I, Data Filtering Common Mixed-Initiative Human Interface Operational High Level Mission Picture Requirements/ROE Management of 5 -10 Ux. V’s Common Information Management Share Mission Relevant Info Report Status Multi-Vehicle Mission Planning Task Scheduling, Routing, Constraints, Priorities Multiple Types of Vehicle Interfaces/Comms UUV Local Planning & SA UAV Local Planning UGV Locally Controlled, & SA Shares Info USV Local Planning & SA Local Task Negotiation, Information Sharing 2

MOUT Site Demonstration Univ. of Penn. , Georgia Tech. , USC, BBN, Mobile Intelligence Summary • Team planning mechanisms geared towards maximizing communications capabilities in adverse conditions while on the move. • Integrated mission specification capabilities for parsing the tasks of overall mission objectives and mapping them onto heterogeneous Ux. V’s • V&V of autonomous algorithms • Joint with DARPA Completed Demo at MOUT Site • Operator tasks multiple types of UAV’s and UGV’s with high-level commands • Mission execution while maintaining communication constraints Accomplishments • Integration of large number of heterogeneous Ux. V’s with different lower-level autonomy software • Framework integrating communications, perception, and execution for Ux. V’s • New algorithm development & implementation for communication sensitive behaviors, heterogeneous Ux. V tasking, distributed SA, and V&V of autonomous systems Future Work • Case-Based Reasoning & Multi-Ux. V Task Allocation to support mission specification • Extend V&V approach to test IA systems 3

Risk-Aware Mixed-Initiative Dynamic Replanning Demo Draper Laboratory/CRA N Region 3 Region 4 Region 2 Keep-out No Comms Target region Region 1 Completed Dynamic Replanning Demo w/ high-fidelity UUV simulation • Single UUV, UAV as comm link • Operator provides high-level tasking/constraints • Autonomously generates plan of activities for UUV to start the mission, transit, search shoreline for particular target classes, end mission • Monitors execution Summary • Single-operator mission management of multiple heterogeneous unmanned vehicles • Extend UUV software w/ increased autonomy onboard UUV & increased dynamic retasking capability for missions w/ comms constraints • Integration with on-board vehicle sensors & external systems Accomplishments • Initial software design & implementation • Integration of initial versions of SA, situation assessment, mixed-initiative interface, control station planning, UUV on-board planning/SA. • Initial integration with existing UUV lower-level autonomous control software • Usability analysis by NAVAIR & NSWC to recommend improvements Future Work • Extend to increase UAV/UUV cooperation, complexity of tasking, integration with other systems, & realism of simulation • In-water demonstration 4

Mission Control for Multiple Ux. V’s Demonstration BAE Systems, Aptima, MIT, BU, Univ. of Minnesota Summary • Allocates tasks based on operator high-level team tasking and constraints • Determines team & individual vehicle tactics to achieve objectives • Schedules activities for heterogeneous resources • Inputs to lower-level vehicle planners • Cooperation w/ realistic comms limitations Multi-Ux. V Simulation Demonstration • Firescout, J-UCAS, BAMS, USV • Operator provides high-level team tasking, ROE’s, & constraints • Replans following changes in environment or new tasking Accomplishments • Extended Mission Control System (MCS) to provide tasking and routing for naval autonomous systems. • Allocates tasks to vehicles including cooperative search and data collection • Constraints include time windows, precedence constraints, and no activity zones • Tasks prioritized as mandatory, high, med, or low. Drops lower priority tasks if not feasible. Future Work • Simulation Demonstration in communication limited environment (joint with Air Force) • Local planning under constraints on vehicle when outside of communication 5

Maritime Image Understanding Northrop/Carnegie Mellon University Completed In-Water Demo on Spartan USV Surface Object Detection & Tracking Shoreline Detection Shoreline Man-Made Object Detection Summary • Maritime Image Understanding for autonomous sensor-directed dynamic replanning • Supports low elevation UUV mast with no mechanical stabilization of image • Detects, classifies, and tracks targets • Joint with DARPA Accomplishments • Reliable surface object detection in clear and hazy conditions • Image stabilization using software only • Low false alarm rate for shoreline man-made object detection • Interesting object detection to direct data collection • Real-Time maritime scene segmentation/high speed video array & limited classification capability Future Work • Integration with on-board dynamic replanning software to enable replanning with sensed data • Additional in-water experimentation • Improved robustness 6

Intelligent Control & Autonomous Replanning of Unmanned Systems (ICARUS) - Lockheed com hellfi SAR 1 EO eng Completed Simulation Demonstration & Evaluation by 2 Navy & 2 Marine Corps Operators with 7 UAV’s (Firescout, BAMS, Future ISR) • Replanning for Dynamic Mission Events • New/Dropped Mission Tasks • Change in order of tasks • Failures/Weather/changes in environment Summary • Integrated suite of components to enable rapid highly automated and fully autonomous mission planning/replanning by high-level objectives • Determines optimized route that meets all constraints & mission objectives while also optimizing secondary priorities Accomplishments • Integrated Multi-UV Task Allocation, Replanning, Replan Assessment, Information/Alert Management, Operator Interface, & Lower level GNC components • Objectives incorporated include search for stationary & mobile targets, EO/IR, SAR, Loiter, Steer-Point, Communication • Constraints incorporated included no-fly zones and LOS comms requirements (moving & stationary) Future Work • Integration with naval control station • Simulation Demonstration in warfare environment at NAVAIR • Increased robustness/integration 7